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In a recent paper [Stud. Hist. Phil. Mod. Phys. 36, 355 (2005)] it is argued that to properly 
understand the thermodynamics of Landauer's principle it is necessary extend the concept of logical 
operations to include indeterministic operations. Here we examine the thermodynamics of such 
operations in more detail, extending the work of Landauer to include indeterministic operations and 
to include logical states with variable entropies, temperatures and mean energies. We derive the 
most general statement of Landauer's principle and prove its universality, extending considerably 
the validity of previous proofs. This confirms conjectures made that all logical operations may, in 
principle, be performed in a thermodynamicaily reversible fashion, although logically irreversible 
operations would require special, practically rather difficult, conditions to do so. We demonstrate 
a physical process that can perform any computation without work requirements or heat exchange 
with the environment. Many widespread statements of Landauer's principle are shown to be only 
special cases of our generalised principle. 
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I. INTRODUCTION 

Landauer's principle holds a special place in the ther- 
modynamics of computation. It has been described as 
"the basic principle of the thermodynamics of informa- 
tion processing" 1] . Yet the literature on Landauer's 
principle is focused almost exclusively on a single, logi- 
cally irreversible operation and a particular physical pro- 
cedure by which this operation is performed 1 . 

In this paper we seek to analyse the form of Landauer's 
principle in a more general context, building upon the 
consideration of the thermodynamics of indeterministic 
logical operations [3, [3|- We will explicitly be considering 
situations where logical states do not necessarily have 
uniform mean energies, entropies or even temperatures 
and we will work in a framework in which logically re- 
versible and irreversible and logically deterministic and 
indeterministic operations can be treated on an equal 
footing. Once we have done this we will have a single 
framework in which the different aspects of Landauer's 
principle can be united. Doing so will help to address 
criticisms 0, [f| of the limited validity of previous proofs 
of Landauer's principle, and criticisms^] of the conclu- 
sions of [U]. 

This will lead us to the following generalisation of Lan- 
dauer's principle: 

Generalised Landauer's principle 

A physical implementation of a logical trans- 
formation of information has minimal expec- 



tation value of the work requirement given 
by: 

(AW) > (AE) - TAS (1) 

where (AE) is the change in the mean inter- 
nal energy of the information processing sys- 
tem, AS the change in the Gibbs-von Neu- 
mann entropy of that system and T is the 
temperature of the heat bath into which any 
heat is absorbed. 

The equality is reachable, in principle, by any 
logical transformation of information, and if 
the equality is reached the physical imple- 
mentation is thermodynamicaily reversible. 

We start by considering what we mean by a logical 
state, a logical operation and the requirements for a phys- 
ical system to be an embodiment of such an operation. 
We will be considering only the processing of discrete, 
classical information here, although we will be assuming 
the fundamental physics is quantum 2 . 

We then construct an explicit physical process, based 
upon the familiar "atom in a box" model, that imple- 
ments a generic logical operation. Thermodynamicaily 
optimising this model will, in general, require considera- 
tion of the probability distribution over the input logical 
states. As this probability distribution is also required to 
quantify the Shannon information stored in the system, 
we will refer to the combination of the logical operation 
and the probability distribution as a logical transforma- 
tion of information. The optimal implementation, using 



'Electronic address: omaroney@perimeterinstitute.ca 
1 Although there little consensus on the naming of these, we will 
refer to the logical operation as RESET TO ZERO (or RTZ) and 
the widely used physical process which embodies this operation 
will be referred to as Landauer Erasure (or LE). 



2 The analysis would proceed largely unchanged for classical 
physics, but it would be unnecessarily cumbersome to attempt 
both. See 0] for a classical treatment. 
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the "atom in a box" physical process, shows that the 
above limit is reachable in principle. We then demon- 
strate that this limit cannot be exceeded by any system 
evolving according to a Hamiltonian evolution. 

We then consider in more detail the implications of this 
limit, including several special cases that correspond to 
more familiar expressions of Landauer's principle, when 
the physical implementation conforms to a set of condi- 
tions which we refer to as "uniform computing" . We will 
show a less familiar set of conditions, which are never- 
theless physically possible, which we call "adiabatic equi- 
librium computing" , and which can embody any logical 
operation without either exchanging heat with the en- 
vironment or requiring work to be performed. We con- 
clude that any logical transformation of information can 
be performed in a thermodynamically reversible manner. 
As this conclusion may seem surprising, we discuss some 
of the practical barriers to achieving this and the partic- 
ular problems presented by logically irreversible opera- 
tions. 



II. LOGICAL STATES AND OPERATIONS 

Although Landauer's principle is about the thermody- 
namics of information processing, very little of the lit- 
erature surrounding it attempts to define what is meant 
by a logical operation and what arc then the minimal re- 
quirements of a physical system for it to be regarded as 
the embodiment of a logical operation. Without first an- 
swering this question, it cannot be certain that the most 
general relationship between information and thermody- 
namics has been discovered. In this Section the abstract 
properties of logical states and operations will be consid- 
ered. This leads to constraints upon a physical system 
which is required to embody the logical states and oper- 
ations. 



A. Logical States 

A logical state simply consists of a variable a, which 
takes a value from a set {1, . . . , n}. If the variable a 
takes the value x then this means that the logical propo- 
sition represented by the statement a — x is true. This 
paper will consider only classical information processing 
on finite machines. This produces additional properties, 
whose assumption is usually implicit 3 : 

1. The set of values is a finite set (and by implication, 
discrete). 



For analog or quantum information processing some of these as- 
sumptions can be relaxed. We will not consider the consequences 
of this here. 



2. The values are distinct. In any given instance the 
variable takes one, and only one, of the possible 
values. 

3. The values are distinguishable. In any given in- 
stance the value taken by the variable can be as- 
certained. 

4. The values are stable. The value taken by the vari- 
able cannot change except as a result of a logical 
operation. 



B. Logical Operations 

A logical operation LOp maps input logical states from 
the set {a} to output logical states from the set {/?}: 

LOp : a -> (3 (2) 

The number of input and output states need not be 
the same. The output states from one logical operation 
may be used as input states to another logical opera- 
tion. Tables U and |TT] show the maps for two of the most 
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TABLE II: Reset To Zero 

commonly 4 encountered logical operations that act upon 
two input states and 1, the NOT operation and the 
Reset To Zero (RTZ) operation. These rules can be rep- 
resented by: 

NOT : -> 1 
NOT : 1 -> 
RTZ : {0,1} -> (3) 

where use has been made of the fact that the RTZ op- 
eration transforms both input states into the output 
state. The RTZ operation is logically irreversible: 



4 There is an even more trivial logical operation: logical Do Noth- 
ing IDN . Including this as a logical operation is not a trivial 
step, as this is the identity operator! It must also be included as 
a time delay operator when one considers a sequence of logical 
operations. 
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We shall call a device logically irreversible if 
the output of a device does not uniquely de- 
fine the inputs. [|[ 

If multivalued maps(see Q [Section 6.1] for example) are 
to be considered, it is necessary to also define logically 
indeterministic 5 computation: 

We shall call a device logically indeterminis- 
tic if the input of a device does not uniquely 
define the outputs. 
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TABLE III: Unset From Zero 
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TABLE IV: Randomisation 

Logically indeterministic operations such as Unset From 
Zero (UFZ) and Randomise (RND) are given in Tables 
IIIII and IIV1 which follow the rules 

UFZ : -> {0,1} 
RND : {0,1} -> {0,1} (4) 

UFZ is logically reversible, while RND is logically irre- 
versible. 

Logically indeterministic operations are perhaps less 
commonly encountered than logically deterministic op- 
erations, and it has been questioned whether these are 
really logical operations([6j, for example, take it as part 
of the definition of a logical operation that it be a single 
valued map). We include them for a number of reasons: 

1. Most importantly, such operations play a signifi- 
cant role in the theory of computational complexity 
classes for actual computers. The complexity class 
BPP, ( Bounded- error Probabilistic Polynomial- 
time), represents a class of computational problems 
for which the inclusion of logically indeterministic 



5 In 0| the term 'non-deterministic' was used. Unfortunately, this 
term has a specific usage in computational complexity classes, 
which does not quite correspond to that used here. To attempt 
to avoid confusion, we have changed our terminology to 'inde- 
terministic'. In terms of computational complexity classes, this 
is closest to probabilistic computation. We hope this does not 
simply introduce more confusion! 



operations can produce an accurate answer expo- 
nentially faster than any known algorithm consist- 
ing only of logically deterministic operations (see 
|12| [Section 3.2.2], for example). Excluding them 
excludes a genuine class of computational proce- 
dures; 

2. By including them we are able to derive a more co- 
herent general framework for the thermodynamics 
of computation. Excluding them creates an arti- 
ficial asymmetry and physical properties ascribed 
to logically irreversible operations in the literature 
may be artefacts of the asymmetry caused by this 
exclusion; 

3. Logically indeterministic transformations of infor- 
mation involve the use of probabilistic inferences. 
There is a point of view, [l(f, [H[ i that regards prob- 
abilistic inferences as a natural generalisation of de- 
ductive logical inferences; 

4. Finally, there seems no special reason not to in- 
clude them as they form a natural counterpart to 
the concept of logically irreversible operations. Any 
conclusion we can draw that applies to the set of 
all such logical operations must necessarily apply 
to all logically deterministic operations. Including 
logically indeterministic operations in our analysis 
will not invalidate its applicability to logically de- 
terministic operations. 



C. Logical Transformation of Information 

To quantify the information being processed by the 
logical operation, the Shannon information measure will 
be used. This requires the specification of a probabil- 
ity distribution over the input and output states. If the 
logical states input to a computation occur with proba- 
bilities P(cv), then the Shannon information represented 
by the input states is 

H a = -Y J P^)\og 2 P(a) (5) 

a 

During the logical operation these input states are trans- 
formed into output states j3. When an input state may be 
transformed into more than one output state, one must 
specify the probability P((3\a) for each possible output 
state. For logically deterministic operations, specifying 
P((3\a) is trivial as Va 3(3 P((3\a) — 1 (or equivalently 
Va 3(3 [V/3' ^ (3 P((3'\a) = 0]). Specifying all the non- 
zero P((3\a) completely specifies the rules of the logical 
operation. We will therefore take the set {P((3\a)} as 
the definition of a general logical operation. For logically 
deterministic operations, this is the list of all combina- 
tions of input and output states that have conditional 
probability one, which is simply the truth table for the 
operation. 
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After the logical operation, the output states (3 will 
occur with probability 



P(/?) = ^P(/3|a)P(a) 



(6) 



so the Shannon information represented by the output 
states is 



ff /3 = -^P(/?)log 2 P(/3) 





(7) 



When we refer to a logical transformation of informa- 
tion, we will mean a logical operation, acting upon input 
states 6 {a}, which occur with probabilities P(a), which 
transforms the input states to output states {/?} with 
conditional probabilities P((3\a). 

The conditional probability that a given output state 
(3 was generated by the input state a is: 



P{a\(3) 



P(q)PC3|q) 



(8) 



and the joint probability that there was an input state a 
and output state (3 is 



P(a,/3)=P(a)P(/3|a)=P(/3)P(a| 



(9) 



This gives an equivalent formulation of logical determin- 
ism and logical reversibility: 

A logically deterministic computation is one 
for which 

Va,/3 P{(3\a) G {0,1} (10) 

A logically reversible computation is one for 
which 

Va, (3 P(a\(3) G {0, 1} (11) 

This is defined in terms of the set {P(a\[3)}. A logical 
operation has been defined only by the set {P((3\a)}, 
with the P(a|/3) dependant upon the input logical state 
probabilities P(a). 
From 

P(a\f3)=0 => P{a,(3) = P{f3\a) = 
P{a\(3) = 1 => P(a' ± a\(3) = 

P{a' ^a,(3)= P(f3\a' ^ a) = 0(12) 

there is an equivalent definition of logically reversible 
computations. An operation is logically reversible, if and 
only if, 



V/3 [P{(3\a) ^0=> [Va' ^ a P(f3\a') 



(13) 



This definition is now independant of the input probabil- 
ity distribution. 

We summarise these properties and some conse- 
quences. 



For simplicity, input states for which P(a) = 0, i.e. which are 
certain to not occur, will not be included in this set. 



1. Logically deterministic operations 

Va, (3 P{(3\a) G {0,1} 
Va [P(a|/3) ^ [V/3' ^ (3 P{a\(3') = 0]] 



Va,/3P(a|/3) G 0, 



P(a)l 
PCS) J 



(14) 



In the case where a particular a — > /3 transition has 
P(/3|a) = 1, we may refer to this as a logically de- 
terministic transition, even if the overall operation 
is not logically deterministic. 



2. Logically reversible operations 

Va,/3P(a|/3) G {0,1} 
V/3 [P(/3|a) ^ [Va' ^ a P(/3|a') 



0]] 



Va,(3P(f3\a) 



0. 



P(a)J 



(15) 



In the case where a particular a — > /3 transition 
has P(a|/3) = 1, we may refer to this as a logically 
reversible transition, even if the overall operation 
is not logically reversible. 



D. Physical Representation of Logical States 

We will now consider what the properties above imply 
for the physical embodiment of logical states and opera- 
tions upon them. The physical system will have a state 
space of possible microstates {fi}. How can these be used 
to embody the logical states? 

1. A particular logical state a will be identified with 
a set of microstates {/i Q } in the state space, in the 
sense that when the physical state of the system is 
one of the microstate [i G {/i a }, then the logical 
state takes the value a. 

2. As logical states are distinct, a given microstate 
can be identified with one, and only one, input 
state. Each set of microstates {f. a } is therefore 
non-intersecting with any other such set of mi- 
crostates: 



{H a }f){v a '^ a } 



(16) 



3. For the logical states to be distinguishable, it is 
necessary that it is possible to ascertain the set to 
which the microstate belongs. We are not consider- 
ing analogue information processing, so the phys- 
ical interactions must not need to be sensitive to 
arbitrarily close (using a natural distance measure) 
states in state space. We replace the point in state 
space /i with the neighbourhood of that point P(^). 
The logical state a is now identified with the re- 
gion of state space corresponding to the union of 
all the neighbourhoods {i?(/u Q )}. The neighbour- 
hoods corresponding to different logical states must 
be non-overlapping. 
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the initial probability distribution of the ensemble 
of microstates within the logical state a. In gen- 
eral, however, we may need to be sensitive to the 
initial probability distribution p a over microstates 
corresponding to the logical state a. 

3. The complete statistical state of the logical process- 
ing system input to the logical operation is 

£p(«K (18) 

a 

where 

Va, K a p a K a = p a (19) 

4. The complete statistical state of the logical pro- 
cessing system output from the logical operation is 
required to be 



4. We can now identify the proposition for the logical 
state a with the projector K a onto the region of 
state space {R(p a )} 

K K , = 5 ,K 

= 1 

a 

K a [R(fi a )] = R(p a ) 

K a [R{n a >^a)] = (17) 

The proposition a is true if the state p, is in the 
region of state space {R(p a )} projected out by K a . 

5. For the physical representation of the logical states 
to be complete, then it must also be the case that if 
the state p is in the region of state space {R(p a )} 
projected out by K a , then the logical proposition 
corresponding to the logical state a is true. 

6. For the logical states to be stable, then under the 



normal evolution of the system, a microstate within 
the region of state space corresponding to a given 
logical state must stay within that region of state 
space 7 . The normal evolution of the system is, 
trivially, a physical embodiment of the 'logical Do 
Nothing IDN' operation. 



E. Physical Representation of Logical Operations 

During the normal evolution of a system, logical states 
do not change. To perform non-trivial logical operations 
new interactions must alter the evolution of the state 
space. All the essential characteristics of a logical oper- 
ation are included in the set {P(/3\a}}. It follows that a 
physical process is an embodiment of a logical operation 
if, and only if, the evolution of the microstates in the 
physical process are such that, over an ensemble of mi- 
crostates in the region {R(p a )} 7 the probability that the 
microstate ends up in the region {R(p )} is just P(/3\a). 

1 . We will assume that the laws of physics are Hamil- 
tonian. The evolution of microstates over the state 
space of the combined system of the logical pro- 
cessing apparatus and the environment must be de- 
scribed by a Hamiltonian evolution operator. 

2. If the interaction of the microstates of the system 
and the environment are such that any individual 
microstate p a starting in state a is randomised so 
that it ends up in the output state (3 with probabil- 
ity P(/3\a) then we do not need to be sensitive to 



A weaker condition, acceptable for most practical needs, is that 
the probability of the microstate leaving the region of a given 
logical state, during the time scale of the information processing, 
must be very low. 



E P ^W (20) 



where 

Vp,K p K = p (21) 

We have not considered separate systems for the logi- 
cal input states, the logical processing apparatus or the 
output states. At first, this seems to assume that the 
system embodying the logical input states must be the 
same as the system embodying the logical output states 
and that the logical processing apparatus cannot have 
internal states - which would seem to be quite a strong 
restriction. This is not the case. Let us consider the case 
where there are three distinct systems: the input state 
system, with states {p a }' : an output state system, with 
states {p }', and an auxiliary system corresponding to 
all internal and external components of the process, with 
states { P a pp}. 

The statistical state is described at the start of the 
operation by 

a 

where we have assumed that the input state system is 
initially uncorrelated to the apparatus but have not as- 
sumed the output state system is initially uncorrelated to 
the internal states {p pp } of the apparatus. The effect 
of the operation would be to evolve the combined sys- 
tem into some new correlated state, combining the three 
systems: 

J2P(a,f3)p' a}0 ®p^ p ®p (23) 

Our approach here is then to consider the state space 
of the combined system of input, output and apparatus 
as a single state space, with input states for a of: 

p a ®(j^w pf p ® P0 ) (24) 
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and output states for (3 of: 

£P(a,/3K i/3 ®^j) ® P0 (25) 

a J 

We then consider a Hamiltonian evolution on the com- 
bined state space to be the operation. This cleanly sepa- 
rates the logical states, embodied by the physical state of 
the combined state space, from the logical operation, em- 
bodied by the Hamiltonian evolution on that state space. 
We have not restricted ourselves by the assumption of a 
Hamiltonian evolution on a single state space, as we have 
the full generality of all possible Hamiltonian interactions 
allowed between the input state system, the output state 
system and the logical processing apparatus. We have 
avoided, on the other hand, any need to consider the re- 
strictions and complications that would arise if we con- 
structed models based upon specific assumptions as to 
how the input state, output state and logical processing 
apparatus systems are allowed to interact. This com- 
pletes the physical characterisation of logical states and 
operations. 

F. Logical vs Microscopic Determinism and 
Reversibility 

There is one final issue that needs to be stated, for 
the sake of clarity, regarding the (absence of a) relation- 
ship between logical and microscopic indeterminism and 
irreversibility. 

Logical indeterminism docs not imply or 
require the existence of any fundamental in- 
determinism in the microscopic dynamics of 
the physical states. Neither is logical de- 
terminism incompatible with the existence 
of fundamental indeterministic dynamics. 

A specific microstate from the input logical state may 
evolve deterministically into a specific microstate of an 
output logical state, while the operation remains logically 
indeterministic, provided the set of the input microstates 
corresponding to the same input logical state do not all 
evolve, with certainty, into microstates of the same out- 
put logical state. 

A specific microstate from an input logical state may 
evolve indeterministically into a number of possible mi- 
crostates, while the operation remains logically determin- 
istic, provided that the set of the input microstates corre- 
sponding to the same input logical state can only evolve 
into microstates from the set corresponding to the same 
output logical state. 

Logical irreversibility does not imply or 
require the existence of any fundamental ir- 
reversibility in the microscopic dynamics of 
the physical states. Neither is logical re- 
versibility incompatible with the existence 
of fundamental irreversible dynamics. 



A specific microstate from the input logical state may 
evolve reversibly into a specific microstate of an output 
logical state, while the operation remains logically irre- 
versible, provided the set of the microstates correspond- 
ing to the same output logical state have not all evolved, 
with certainty, from microstates of the same input logical 
state. 

A specific microstate from an input logical state may 
evolve irreversibly into a specific microstate, while the 
operation remains logically reversible, provided that the 
set of microstates corresponding to the same output log- 
ical state can only have evolved from microstates in the 
set corresponding to the same input logical state. 



III. THERMODYNAMICS OF LOGICAL 
OPERATIONS 

We now undertake the main task of this paper: to 
determine the limiting thermodynamic cost to a logical 
operation. We will do this in two steps. 

Firstly, we will construct a physical process, capable of 
implementing any logical operation, as we have defined 
them, and we will consider the optimum thermodynamic 
cost to the process. This optimum will be considered 
in two ways: for individual transitions between specific 
logical states; and as an expectation value over an en- 
semble of operations. Both work required to perform the 
process and heat generated by the process will be cal- 
culated, where it is assumed that all heat generated is 
absorbed by a heat bath at some reference temperature 
Tr. 

To calculate the expectation values, we must consider 
the probability distribution over the input logical states. 
For this we will use the probability distribution used to 
calculate the Shannon information being processed. The 
optimum process will, of necessity, involve various ideali- 
sations (such as frictionless motion and quasi-static pro- 
cesses) that cannot be achieved in practice. The purpose 
is to demonstrate not that it is possible to build such 
optimal operations, but rather that there is no physi- 
cal limitation, in principle, on how close one can get to 
them. 

Then we will prove that there cannot exist any physical 
process that can implement the same logical transforma- 
tion of information, but with a lower expectation value 
for either the work requirement or the heat generation. 
The optimum process, for our particular implementation 
of a logical transformation of information, is also the op- 
timum for any possible implementation of that transfor- 
mation. 



A. Statistical mechanical assumptions 

We will now clearly state the statistical mechanical as- 
sumptions that are being made. There are a number of 
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different approaches to the foundations of statistical me- 
chanics and, as the models discussed here involve such 
idealisations as the treatment of individual atoms, it is 
important to be clear which approach is being taken. 
In this article we will assume the standard structure of 
Gibbs canonical statistical mechanics: we will be deal- 
ing with Hamiltonian flows with probability distributions 
over a state space, we will assume that a system that 
has been thermalised can be represented by a canoni- 
cal distribution over its accessible state space, and will 
be initially statistically independent of any other system. 
While these assumptions are clearly open to debate, a full 
discussion or justification of them lies outside the scope 
of this article (although see fl3||). 

1 . The system consists of the logical processing appa- 
ratus (including auxiliary systems as discussed in 
Section Hi E[) and a number of heat baths. A heat 
bath is simply a system that has been allowed to 
thermalise at some temperature and is sufficiently 
large that any energy transfer with the logical pro- 
cessing apparatus will have negligible effect upon 
the heat baths internal energy. The Hamiltonian 
for the combined system is: 



H = H L + Y^Hi + Vi) 



(26) 



where Hl is the internal Hamiltonian for the log- 
ical processing apparatus, Hi the internal Hamil- 
tonian of the heat bath i and Vi is the interaction 
Hamiltonian between the logical processing appa- 
ratus and the heat bath i. We assume there is no 
interaction between heat baths. The density ma- 
trix of the combined system is pc, and ph is the 
marginal density matrix after tracing over the heat 
bath subsystems. 

2. Work is performed upon the apparatus through the 
variation of some externally controlled parameter 
X, which affects the energy eigenvalues and eigen- 
states 8 . 



H L {X) =Y J E n {X)\E n {X)) (E n (X)\ 



(27) 



The mean work performed, as the parameter is var- 
ied from Xq to X\ is given by: 



AW 



Tr 



A'n 



dH L {X) 
OX 



Pl(X) 



dX 



(28) 



Note that the density matrix pl may be varying as 
X varies. We will assume that neither the internal 
Hamiltonians of the heat baths nor the interaction 



More generally, one should consider a number of controllable 
parameters, which are each varying in time. 



Hamiltonians have controllable parameters: work 
is only performed upon the logical processing ap- 
paratus itself. 

3. The mean change in internal energy of the logical 
processing apparatus is 

AE = Tr [H L (X 1 )p L (X 1 )} - Tr [H L (X Q )p L (X )] (29) 

4. If we now assume negligible changes in interaction 
energies: 

Vi, Tr WipciXt)] « Tr [V iPc (X )] (30) 



then 



AW -AE = Y AQi 



(31) 



where 

AQi = Tr [H iPi {Xi)] - Tr [H iPi (X )} (32) 

is the increase in internal energy of the heat bath 
i, and P i is the marginal density matrix of the heat 
bath, after tracing over the logical processing ap- 
paratus and all other heat baths.. 

5. If the evolution of density matrix is such that it al- 
ways remains diagonalised by the energy eigenstate 
basis, so that 



PL (X) = J2Pn(X) \E n (X)) (E n (X) | 



(33) 



then: 



AW 



EAft = / E 



X, 



x 

X: 



52Pn(X)^±dX CU, 



OpnjX) 

dX 



dX 



E n (X)dX (35) 



It is important to note that these [5] points make 
no assumption regarding the identification of either 
thermodynamic entropy or thermal distributions. 
Neither the canonical distribution nor the Gibbs- 
von Neumann entropy has been used. 

The following results depend upon the assumption 
that a heat bath is represented by a canonical dis- 
tribution and that a limiting ideal case exists of 
thermalisation through a succession of brief inter- 
actions with small subsystems of a heat bath. The 
calculations are well known (see [H, 03, HU , for ex- 
ample) and the results are stated here purely for 
clarity. No formal identification of the Gibbs-von 
Neumann entropy with thermodynamic entropy is 
required to derive these results. 
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6. A system that is brought into contact with an ideal 
heat bath will, over time periods long with respect 
to its thermal relaxation time, be well represented 
by a canonical probability distribution 



-H/kT 



Pa 



TV [e 



-H/kT] 



(36) 



over accessible states of the system, with the T be- 
ing the temperature of the heat bath, and H the 
Hamiltonian of the system over the accessible sub- 
space. 

7. In the limit of isothermal quasistatic processes, the 
system is in contact with an ideal heat bath at some 
temperature, and system stays in thermal equilib- 
rium with the heat bath at all times. 

8. In the limit of adiabatic quasistatic processes (or 
essentially isolated[14( processes) the system always 
remains in a (canonically distributed) thermal state 
but there is zero mean energy flow out of the system 
(AW — AE). The temperature of this state may 
vary. 

9. We will assume that the only systems with which 
the information processing system interacts are 
ideal heat baths at temperatures {T Q }, {Tjg} and 
Tr, and a work reservoir, and that there are no ini- 
tial correlations between the system and the heat 
baths. 

While these assumptions involve significant idealisations, 
they are the kind of idealisations that are standard in 
thermodynamics and statistical mechanics. Rather than 
representing a physically achievable process, they repre- 
sent the limit of what can be physically achieved. There 
is no physical reason why one cannot, in principle, get 
arbitrarily close to these results. 

Although the value of the Gibbs-von Neumann en- 
tropy, — fcTr [pin [p]) will be calculated for the input and 
output logical states, all results in this Section, in terms 
of work required and heat generated, are derivable, from 
the assumptions stated, without needing to identify this 
property with thermodynamic entropy 9 . 



B. Generic logical operation 

1. Input logical states 

We start the operation with the logical states repre- 
sented by physical states with the properties: 



9 See [lijll and in 0] [Chapter 6] where this kind of calculation is 
carried out in detail for same the kinds of systems considered 
here. 



1. An input logical state, a, to the logical computa- 
tion is physically embodied by a system confined to 
some region of state space. The distribution over 
the microstates of that region gives the density ma- 
trix p a . 

2. p a has mean energy E a — Tr [Hi,p a ] = Tr [H a p a ], 
where H a = K a HLK a . 

3. For simplicity, in the main section, we will assume 
the input logical state a is canonically distributed, 
as if it has been thcrmalised with a heat bath at 
temperature T a . 



-H a /kT a 



Tr [ e -H«/kT a j 



(37) 



This assumption is not essential, and can easily be 
relaxed without affecting any result. If the initial 
density matrix is not a canonical distribution, then 
it is possible to construct a unitary operator that 
acts upon the system in isolation and rotates it into 
a canonical state, with neither heat nor work re- 
quirement. 

To give an explicit construction, suppose the initial 
Hamiltonian and density matrix are H a 1 ' and p& , 
such that 



Tr e - H ^/ kT « 



(38) 



for any T a . Given the diagonal representation 

n 

then the Hamiltonian, acting between < t < r, 



id 



cos — -sin — 
2t t 



Kt 



Tit 



sur,-,- sin - t 



Tit 



2ih 9 / 7rf , 
sin — I In 

T \ T 



with 

H a = ^2 E n |7„) (7„ 



(40) 



(41) 



E n = E a - kT a \ ln(p n ) - J2 Pm ln(p m )j (42) 

varies continuously from H a ^ to H a , and has the 
effect of leaving the system, after time r, in the 
stationary canonical state 



-H a /kT c 



Pa 



Tr 



(43) 
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p a is unitarily equivalent to pa and TV [H a p a ] 



Tr 



tl a p a 



The mean work requirement is zero 

and no heat is exchanged with the environment. It 
should be noted that this construction holds even 



if pa is not diagonalised in the eigenstates of H a ' 

4. The Gibbs-von Neumann entropy of the input log- 
ical state is: S a = — fcTr [p a In [p a ]]. 

5. There are M possible input logical states. 



(i) 



2. Output logical states 

The output logical states may be similarly charac- 
terised by: 

1. An output logical state, /3, from the logical compu- 
tation is physically embodied by a system confined 
to some region of state space. The distribution over 
the microstates of that region gives the density ma- 
trix pp. 



2. pp has mean energy Ep = Tr [H' L pp] = Tr H' pp , 



where Hp 



KpH' T K 



a- 



3. Again, for convenience we will assume the output 
logical state j3 is canonically distributed as if it has 
been thermalised at a temperature Tp. This will 
make the density matrix: 



P0 



Tr e~ H 'f>/ kTf> 



(44) 



Again, this assumption is easily dropped. If the 
final state is required to be a non-canonical density 

y , then p^P can 



matrix p^ , 



with Hamiltonian H 



be obtained from pp by constructing H'^ and Hb 
in the same manner as H a and Ha above. 

4. The Gibbs-von Neumann entropy of the output log- 
ical state is Sp = —fcTr [pp In [pp]]. 

5. There are N possible output logical states. 

6. The output logical state (3 must occur with proba- 
bility P{(3\a) given input logical state a. 



C. The transformation of information. 

If we consider the actual microstate of the system, 
within the region corresponding to a given logical state, 
as being some free 10 parameter, then the physical repre- 
sentation of each logical state may initially be regarded 
as a potential well with some arbitrary shape, such that 
the free parameter is confined within the well. The po- 
tential wells associated with different logical states are in 
different regions of physical space, separated by high po- 
tential barriers, such that there is a very low possibility 
of transitions between different logical states. 

This can be represented as an atom in one of a num- 
ber of boxes, where the free parameter is the location of 
the atom in the box. The logical state is represented by 
the particular box, or potential well, within which the 
atom is confined. The physical transformation of the in- 
formation will take place in nine steps. Steps 1 through 
to 3 will bring the input logical states into standardised 
physical states at a shared reference temperature. Steps 
4 is the logically indeterministic implementation of the 
P{j3\a) transition. Step 5 and Step 6 implement the join- 
ing together of the (3 output states from the different a 
input states, giving the logically irreversible stage. Steps 
6 through to 9 then alter each output logical state to the 
required final physical state. 

Calculations for work requirements, heat generation 
and so forth, follow the statistical mechanical calcula- 
tions above. Particularly detailed calculations for 'atom 
in a box' type systems are considered in references such 
as @, [ljl Q3, [l8j]. The key results can be summarised. 
The Hamiltonian for an infinite square well potential, of 
width I, holding an atom of mass m is 



2^2 



8ml 



(45) 



Work is performed upon the system by varying the I pa- 
rameter (width of the box) . 

In a canonical thermal state at temperature T, the 
mean energy is 



E = 



E li 71 
n 8ml 



roe !m|2 ' T 



y 



g 8ml 2 kT 



(46) 



and the Gibbs-von Neumann entropy is 



We will also note here that probabilities enter the cal- 
culation at two levels: as a probability distribution over 
the microstates within a given logical state, and as a 
probability distribution over the different logical states. 
We will attempt to keep these formally separate. From 
this point onwards, the microstate probability will be rep- 
resented only by the density matrix. Explicitly appearing 
probabilities and averages will always refer to the prob- 
ability distribution over the logical states. 



S 



2 



iml 2 kT 

2emkTl 2 ~ 



J2n e 



irh 2 



Not externally controlled. 



fc In 



E 



(47) 
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The approximations hold when the temperature is high 
with respect to the ground state: 



wti 2 
2em 



(48) 



1. The first step is to continuously and slowly deform 
the potential well of each separate logical state into 
a square well potential. 

7(1) 



The square well should be deformed to width d, 



irh 2 



a s a /k 



2emkTr, 



(49) 



where m is the mass of atom. 

This state has mean energy and entropy 

1 
— / 

2 

5n 



e(i) 



- k T, y 



(50) 



If this deformation is carried out sufficiently slowly, 
the mean heat generation is zero and the work re- 
quirement is 



1 



-kT a E a 



(51) 



This is a mean work requirement for the operation. 
Fluctuations may occur around this value. 

The system is may now be pictured as a box, di- 
vided with M — 1 partitions. When the atom is 
located between the a — 1 and a partitions, the 
system is in logical state a. This can be seen in 
Figure QJa). 

Remove the system from all contact with heat baths 
and then, slowly, adiabatically vary the width of 
each square well to d^ : 



dV 




(52) 



At the limit of a slow, quasistatic process, this will 
leave each logical state with a density matrix equal 
to a canonical thermal system with temperature 
Tr. Mean energy, entropy and mean work require- 
ments are: 



E. 



si 2 



(2) _ 



1 



-kT R 



fcln 



' 2emkTji 
irh 2 



W a 2) = -kT R - 
As the total width of the box is now 



L 



(2) 



irh 2 \ 
2emkTn J 



S a >/k 



(53) 



(54) 



(a) 



L 



i- ► 



(b) 



w, L 



P\P\\)w i L p[\\ a \ w r p[B\a\w„L 



(c) 



FIG. 1: Arranging input states 



then 



d^=L 



A/k 



oS a ,/k 



(55) 



3. Now bring the entire system into contact with heat 
baths at the reference temperature Tr. Slowly and 
isothermally move the positions of the potential 
barriers separating the square wells(see Figure Q] 



(a-b) ). Move the i barrier to the position Xi 



(56) 



where J2 Q Wa = 1- ^ ne vames °f w a have not been 
specified. Varying these will be used to optimise 
the operation. 

(3) 

Each logical state now has a width d a = w a L. If 
w a = for one of the input states a this stage will 
compress the volume of that state to zero. Clearly 
this can only be allowed to take place if there is 
no possibility that the partition is occupied by the 
atom! 
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w, L 



E (3) 



Si 3) = fcln 



1 2emkTji 
irh 2 



= fcT fl ln 

,(3) 



d (3) 

~d^ 

d {3) 

da 

3w 



(57) 



where Q a is the heat generated in the heat bath 
if the atom is in the a partition. 



W«J_ 

P(8|aJWaL 



W(« + 1)L 



FIG. 2: Inserting subpartitions 

Insert N — 1 new potential barriers slowly into each 
partition. Within a given partition a, the barri- 
ers should be spaced according to the probabilities 
P(f3\a) of the logical operation (see Figure[2]). They 
have a width 



i (4) 



P(/3\a)w a L 



(58) 



This is the logically indeterministic step of the com- 
putation, in Figure[T](b-c). There are M partitions, 
each with N subpartitions. 

If the atom was located in partition a beforehand, 
and the system has been allowed to thermalise at 
temperature Tr for a period of time greater than 
the thermal relaxation time, then the probability 
of the atom now being located in the (a, 0) sub- 
partition is P((3\a). For logically deterministic op- 
erations, then all non-zero P((3\a) are equal to one 
and no partitions need be inserted. 

The mean energy, and the entropy, associated with 
the atom being located within a particular (a, [3) 
subpartition is: 



1 {4) 

a,8 



'2emkTn 
nh 2 



(59) 



(C) 



(d) 




FIG. 3: Rearranging the partitions 



each a partition, we gather first subpartition, corre- 
sponding to output logical state (3=1, and collect 
them together. Repeat this for each set of (3 sub- 
partitions, from all the a partitions. Finally this 
produces a sequence of N (3 partitions, each with 
M a subpartitions. This is illustrated in Figure [31 



(d) 



(e) 



(f) 



L ' 



5. Now rearrange the subpartitions so that, for each 
/3, all the (3 output partitions are adjacent. From 



FIG. 4: Arranging the output states 
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6. Remove the potential barriers within each f3 output 
partition and leave the box for a time that is long in 
comparison to the atoms thermal relaxation time. 
The (3 partition has a width 



dp =wpL 



(60) 



where 



w fi =Y,™«P(/3\a) (61) 

a 

and if the atom is located in the f3 partition, then 





= ~kT R 
2 


C(6) 


= k In 



7 (6) 



' 2emkT R 
irh 2 



(62) 



This is the logically irreversible stage and is illus- 
trated in Figure 2] (d-e). This stage is trivial for 
logically reversible computations, for which each (3 
output partition is composed of only one a subpar- 
tition, and so has no internal barriers. Note also 
that if Va, P([3\a) — 0, then the atom can never be 
located in the (3 partition. 

7. Now slowly and isothermally resize the output par- 
titions. The barriers should be moved until the (3 
partition has width 



,(7) 



2emkTji 



(63) 



See Figure ffl(e) to (f). 

The overall width of the box may change by this 
operation, and is now: 

J2b' expV /fc 



so that 



" E^expVA 



(64) 



(65) 



For the atom located in the (3 partition, we have: 



E 



(7) 



1 



;kT R 



S { J } = kin 



d {7) 



( l2emkT R \ 



— S n 



Wi 7) = kT R \n 



Q { P = fcTflln 



7(7) 



7 (6) 
Y (6) 



(66) 



(7) 

where Qp is the heat generated in the heat bath. 



Now remove all contact from the T R heat baths. 
With the system thermally isolated, slowly and adi- 
abatic resize the output partitions to the widths: 



d {8) - d (7) 



(67) 



If the atom is in the (3 partition, the effect of this 
quasistatic, adiabatic evolution is to leave the atom 
in a canonical thermal state with temperature Tp. 



E 



(8) 



;-(8) 



w, 



(8) 




\kTp 
Sp 

\kTp - \kT R 



(68) 



9. The output logical states (3 are now all at the re- 
quired temperature, and entropy. For complete- 
ness, bring each separate (3 partition into thermal 
contact with a heat bath at the appropriate temper- 
ature Tp and slowly, continuously and isothermally 
deform the shape of each square well potential into 
the final potential for the output logical state. 



E^ = Ep 



? (9 

4 9 



S ( ° } = Sp 



Wl 9) = Ep 



-kTp 
2 p 



(69) 



This completes the physical implementation of the logical 
operation. 



D. Thermodynamic costs 

The procedure detailed in the previous Section fulfils 
the requirements of a generic logical operation. The input 
logical states are represented by the appropriate physical 
input states, the output logical states are represented by 
the appropriate physical output states, and the transi- 
tions between them occur with probabilities P((3\a). 



1. Individual transitions 

Adding up the work and heat values, across all steps, 
for a system which starts in logical state a and ends in 
logical state (3 gives: 



AW Qi/ j = {Ep-T R Sp)-{E a -T R S a ) + kT R hx 

~wp 



Wp 



AQ a ,p = T R [S a - Sp + k hi 



(70) 
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1. For a logically reversible transition, 



2. Expectation values 



Wp 



P(/3\a) 



(71) 



and so is indcpcndant of the choice of w a . If the 
transition is also logically deterministic, P(/3\a) = 
1 and the logarithmic term is zero. The work re- 
quirements are 

AW a ,p - (Ep - T R Sp) - (E a - T R S a ) (72) 



2. If the logically reversible transition is indeterminis- 
tic, the work requirement is reduced by the quantity 
-kT R In [P(/3\a)]. If P((3\a) is small, this term can 
be large, even to the extent of making the work 
requirement negative (i.e. implying work may be 
extracted from the process) . 

3. Now consider logically irreversible transitions. 
When the transition is logically deterministic, wp 
is the sum of all the w a values where the transition 
is permitted. It is therefore always the case that 
— > 1. This implies an increased work require- 
ment compared to a logically reversible, determin- 
istic transition between equivalent (a, f3) states. 

4. Finally logically irreversible, indeterministic tran- 
sitions may, in principle, take values for ^f- both 
above and below 1. 

Let us consider optimising the thermodynamic cost of an 
individual a — > (3 transition. The only free variables are 
the w a . For logically reversible transitions, these have no 
effect and the cost is always: 

AW a ,p = {Ep-T R Sp)-{E a -T R S a ) + kT R ln[P(0\a)] 
AQ aJj = T R (S a -Sp + kln[P(p\a)}) (73) 

For logically irreversible transitions, the quantity ^f- 
should be made as small as possible, subject to the con- 
straint that J2 a w a — 1. From wp — J2 a > w a'P{P\ot') it 
must be the case that 



wp > w a P(/3\a) 



(74) 



The problem with optimising for an individual transi- 
tion is that this can go catastrophically wrong if the oper- 
ation is performed upon any of the other a! input logical 
states. For logically irreversible processes, as w a > — > 0, 
then AW a ' t/ 3 — > oo. 

We need to consider an optimisation over the full set of 
input logical states, rather than with respect to a single 
input logical state. For the set of all possible transitions, 
we will seek to minimise the expectation value, or mean 
cost, of performing the operation. 

This is not the only criteria that could be used. One 
may seek instead, for example, to optimise by a minimax 
criteria: minimising the maximum cost that might be 
incurred. This would lead to a different set of w a to those 
we will calculate here. The maximum cost that might be 
incurred with such a set would, for certainty, be no higher 
than the maximum cost we will arrive at here. However, 
the expectation value for the cost, with the different set, 
would be at least as high as the expectation value we will 
find. 

To be able to calculate an expectation value, a proba- 
bility distribution over the input logical states is needed. 
For this we will use the probabilities that go into the cal- 
culation of the Shannon information of the input state: 
P(a). The probability of the transition a — > (3 occurring 
is then P(j3\a)P(a) and the expectation values for the 
work requirement is: 

(AW) = J2 P (P) ( E ? - T R S ?) - E P (") ( E « - T ^ 



Equality is reached by setting w a ' — 0, for all the input 
logical states a' ^ a where P(f3\a') ^ 0. This gives wp — 
w a P(f3\a). If the transition is a logically deterministic 
one, ^ = 1, otherwise ^f- < 1 and the work requirement 
is reduced (as for a logically reversible, indeterministic 
transition). The result is similar to logically reversible 
transitions: 

AW a4i > {Ep-T R Sp)-{E a -T R S a ) + kT R ln[P((3\a)} 
&Q a ,p > T R {S a -Sp + k]n[P{p\a)\) (75) 



wp 



W-'n 



+fcT K ^P(a,/3)ln 
where P(a,(3) = P(/3\a)P(a) and P{(3) 



(76) 



For logically reversible transformations, this is fixed: 



^ = P{fi\a) 



P(a) 



(77) 



For logically irreversible transformations, we must vary 
the w a to minimise the function 



wp 



X = ]T />,/?) In 

a, fS 

Consider the similar function 

P a 



(78) 



]T />,/?) In 



a,/3 

X-Y = ]TP(a,/?)ln 

a,0 

= £)P(a,/?)ln 

a, (3 

> 



P(a) 
wpP(a) 



(79) 



P(/3)w a 
P(a,0) 



P(P)w(a[ 



(80) 
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where 



3. Multiple Heat Baths 



w(a\/3) 



P{l3\a)w a 

Wf3 



(81) 



and the equality occurs iff P(a,/3) = P((3)w(a\(3). As Y 
is independant of the values of the w ai then the minimum 
value of X is precisely the value of Y. This minimum 
value of X is reached when w a — P{a) 1 which leads to 

The result can easily be re-expressed as: 



(AW) > ^P^^-TflC^-felnP^))) 



-J2 P ( a ) (E a ~T R (S a -fclnP(a))X82) 



This is the minimum expectation value of the work re- 
quirement for the logical operation, using the physical 
procedure we have described. The same expression holds 
for logically reversible, irreversible, deterministic and in- 
deterministic operations. It is not hard to see that this 
also minimises the expectation value of the heat gener- 
ated: 



(AQ) 



> 



-T R \Y,P(/3) (S - kin P(/3)) 
-J2 P ( a ) (S a -k\nP(a))\ 



(83) 



As was noted for the case of LE in 0] , to achieve the op- 
timal physical implementation of a logically irreversible 
operation requires the physical process to be designed 
for the particular probability distribution P(a) over the 
input logical states 11 . A physical implementation opti- 
mised for one input probability distribution will not, in 
general, be optimised for a different input probability dis- 
tribution. For logically irreversible operations it is only 
possible to thermodynamically optimise the logical trans- 
formation of information (where the input probability is 
specified). Without a probability distribution (even a 
default assumption of equiprobable input states) it does 
not even make sense to talk about optimising the expec- 
tation value for the work or heat requirements, or about 
the Shannon information of the input and output states. 



11 It is worth noting that this is not the same as having a prior 
knowledge of the input logical states. Having prior knowledge of 
which input state occurs allows one, trivially, to do rather better 
than this, by choosing w' a = for all other input states. This 
optimises for all individual transitions that come from the known 
a input state, but requires a different physical implementation 
each time a different input logical state occurs. That different 
physical implementation is, in each case, equivalent to a logically 
reversible operation. 



For completeness, we note that if there are several heat 
baths available, at different temperatures, the equations 
may be easily generalised. Defining: 



(AQ) = ^ (AQi) 

i 

(AO,) 



T 



(AQ,) 



(84) 
(85) 



where (AQi) is the mean heat generated in a heat bath 
at temperature Tj, w e may simply replace Tr with T 
and (AQ) with (AQ), in Equation[82j and all subsequent 
equations. In effect, this is equivalent to the possibility of 
using reversible Carnot cycles to rearrange heat between 
any heat baths available, in addition to performing the 
logical operation with a single heat bath. 

The introduction of multiple heat baths has little prac- 
tical significance though. If 

PW {Sfi -kin PG0))-J^ P{a) {S a - k In P(a)) < 

a 

(86) 

then the least work is required by generating all the heat 
in the coolest heat bath available. If 

p (P) ( s p - k ln p (P))-Y. p ( a ) ( s « - kln p («)) > 

a 

the opposite is true. The least work involves only gener- 
ating heat in the hottest heat bath. 



E. Optimum physical process 

We have shown that a particular physical process can 
implement a logical operation, with a minimum expecta- 
tion value for the work required or heat generated. Per- 
haps other physical processes might exist which can per- 
form the same logical operation at a lower cost? We 
will now prove that no physical process can implement 
the same logical transformation of information at a lower 
cost. 

The initial statistical state of the logical processing ap- 
paratus is 



Pi 



= Y,P(a)P a 



The final statistical state is 



pf = Y j p(p)pp 





(88) 



(89) 



We assume that the environment is initially well de- 
scribed by a canonical thermal state pb(Tr), at tem- 
perature Tr, and that it is uncorrelated with the initial 
state of the logical processing system. 
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Now consider the initial density matrix of the joint 
system of the logical processing system and the apparatus 



P = Pi® Pe{Tr) 



(90) 



From the well known[14j, |l9j, 120J properties of unitary 
evolutions and density matrices: 



Tr[/9ln[p]] = Tr [p 1 In [p'\] 



(94) 



Tr [pin [p]] = Tr [ Pl In [ Pl ]\ + Tr [p E (T R ) In [p E (T R )]] 

For any unitary evolution upon the combined system 
to be a physical representation of the logical state, it must 
evolve the system to some state p' such that the marginal 
distribution of the information processing apparatus is: 

PF = Tt e [p 1 ] (92) 
The marginal distribution of the environment is then: 

p' E = Tr F [p 1 ] (93) 
I 



Tr \ff In [f/]} > Tr [p F In [p F ]} + Tr [p' E In [p' E ]} 
As Pe(Tr) is a canonical distribution 



(95) 



so 



kT R 



\n[p E (T R )]-lnZ 



(96) 



Tr 



PeWp'e] 



He 
kT R 



Tr 



Pe{T r ) ( In [pe{T r ) 



r 



He 



Tr[p' E (ln[p' E ]-ln[p E (T R )])] >0 (97) 



where He is the internal Hamiltonian of the environment. 
A simple rearrangement gives 



Tr [p! In \pi]] - Tr [p F In [p F ]} > 



R 



Tr [H E p E (T R )} Ty[Hep' e ] 
kT R 
(98) 

As the physical representations of the logical states are 
non-overlapping : 

- kTr [pi In [pi]] = ^P(a) (S a - fclnP(a)) (99) 

a 

-fcTr[ Pi ,ln[ PF ]] - ^P(/3)(5 /3 -fcliiP(/3))(100) 

|8 

The expectation value for the work performed upon 
the system must equal 12 the expectation value for the 
change in the internal energy of the system plus the ex- 
pectation value for the change in the internal energy of 
the environment: 

(AW) = Y^P^Ep + TvIHep'e] 

-J2 p (^)E a -Tr[H E p E (T R )] (101) 



12 We assume that the interaction energy between system and en- 
vironment is negligible at the start and end of the operation. 
Both this assumption, and the assumption that the environment 
is initially an uncorrelated Gibbs state, do not appear to hold in 
21, 221. ' 



From this we conclude that, for any physical process, 
which takes input logical states {a} with probabilities 
P(a) and produces output logical states {/?} with prob- 
abilities P(/3), then the expectation value of the work 
requirement for this process cannot be less than 

(AW) > ^P(f3)(E -T R (Sp- kin P(/3))) 

- P(a) {E a - T R (S a -kin P{a) 002) 



There is no physical process that can do better, in terms 
of an expectation value for the work requirement, or for 
the heat generation, than the process developed in Sec- 
tion nm 

We emphasise that the relationships we have derived in 
this section do not depend upon the results of the specific 
process we examined in the previous section. No assump- 
tions are made regarding the details of the physical pro- 
cess which represents the logical operation, beyond the 
requirements that it is a unitary evolution of the com- 
bined state space of system and environment and does, 
in fact, faithfully represent the operation. No assump- 
tions are required about the physical representation of 
the input and output logical states, except those made 
in Section III Dl It is not assumed that the environment 
is an ideal heat bath, is in some thermodynamic limit, 
or is in thermal equilibrium after the operation. The re- 
sults require only that the environment be a canonically 
distributed and uncorrelated system at the start of the 
operation. Given these assumptions, the result follows: 



16 



there is no physical representation of the logical oper- 
ation that has a lower expectation value for the work 
requirements or heat generation. 

IV. GENERALISED LANDAUER'S PRINCIPLE 

There are several different, but formally equivalent, 
ways of expressing the Generalised Landauer's principle 
(GLP). It will be convenient to use the notation: 

(AE) = Y,P(0)Ef>-52P(<*)E a 

a 

AS = ]rP(/?)(^-MnP(/?)) 



-]T/» (S a -klnP(a)) 

a 

AH = - ^ P(J3) log P(f3) + ^P(a) log P($)03) 

(i a 

for: the change in the expectation value for of the inter- 
nal energy of the information processing apparatus; the 
change in the Gibbs-von Neumann entropy of the sta- 
tistical ensemble describing the information processing 
system; and the change in the Shannon information of 
the logical states over the course of the operation. 



information AH between the input and output states. It 
is the change in the Gibbs-von Neumann entropy of the 
logical system, taking into account any changes in the 
entropies of the subensembles that represent the input 
and output logical states. It can be related to the change 
in the Shannon information by 

AS = ^P(i3)S -^P{a)S a +kAH In 2 (107) 

a 

C. Entropic cost 

The change in the Gibbs-von Neumann entropy of the 
environmental heat bath is given by: 

AShb = -kTi [p' E In \p' E ]] + kTr [p E (T R ) In [ P e(Tr)}] 

(108) 

which gives the entropic form of the Generalised Lan- 
dauer's principle: 
GLP3: Entropy 

A logical transformation of information re- 
quires a minimal change in the Gibbs-von 
Neumann entropies of the marginal statistical 
states of an information processing apparatus 
AS and its environment AShb of: 

AShb + AS >0 (109) 



A. Work requirements 

GLP1: Work 

A logical transformation of information has 
a minimal expectation value for the work re- 
quirement given by: 

(AW) > (AE) - T R AS (104) 
B. Heat generation 



This is a trivial consequence of the requirements that 
the evolution be unitary and that the statistical states 
of the logical processing system and the environment 
be initially uncorrelated. The expectation value of the 
heat generated in the environment is at least equal to 
the increase in the Gibbs-von Neumann entropy of the 
marginal state of the heat bath: 

(AQ) > TrAShb (HO) 

This allows us to deduce GLPl or GLP2 from GLP3, 
but not reverse 13 . 



Noting that 



D. Information 



(AQ) = (AW) - (AE) 



(105) 



is equal to the expectation value of the heat generated in 
the heat bath: 
GLP2: Heat 

A logical transformation of information has a 
minimal expectation value for the heat gen- 
erated in the environment of: 



If we define the term: 



AS L = ^P(/3)^-^P(a)5 Q (111) 



we get 



AShb+AS l > -fcAPln2 



(112) 



(AQ) > -TrAS (106) 

It is important to remember that the term AS* appear- 
ing in GLPl and GLP2 is not the change in Shannon 



In the limiting case of an ideal heat bath and quasistatic pro- 
cesses, the equality is reached and the deduction can then go in 
both directions. 
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This expression seems suggestive. If we regard the terms 
AShb and ASl as changes in the entropies of the 'non- 
information bearing degrees of freedom' of the environ- 
ment and the apparatus, respectively, then we appear to 
have provided a quantitative version of Bonnet's state- 
ment that 

any logically irreversible manipulation of in- 
formation [AH] . . . must be accompanied by 
a corresponding [k In 2] entropy increase in 
the non-information bearing degrees of free- 
dom of the information processing apparatus 
[ASl] or its environment [AShb] [l| 

although unlike Bennett, we do not restrict this to irre- 
versible transformations of data. 

This produces what may be taken as the information 
form of the GLP: 

GLP4: Information 

A logical transformation of information re- 
quires an increase of entropy of the non- 
information bearing degrees of freedom of the 
information processing apparatus and its en- 
vironment of at least —A: In 2 times the change 
in the total quantity of Shannon information 
over the course of the operation: 

AS NIB df > -kAH\n2 (113) 

where ASnibdf = AShb + ASl- This is quite gen- 
erally true and follows directly from GLP3 and the defi- 
nition of ASl- 

V. MODELS OF COMPUTING 

We will now discuss some of the consequences that can 
be drawn from the Generalised Landauer's principle by 
varying the thermodynamic properties of the input and 
output states. This allows us to consider the effects of 
having different energies and entropies for the physical 
states that embody the logical states and has some sur- 
prising consequences. 

A. Uniform Computing 

When we make the assumption that the computation 
takes place at the same temperature throughout, such 
that 

Assumption 1: Isothermal 

Va, f3 Tji = T a = Tp (114) 

then we shall call this isothermal computing. 

In the most commonly encountered set of assumptions 
for the thermodynamics of computation, we have, in ad- 
dition to the assumption of isothermal computing, the 



physical states, that represent the logical states, all have 
the same entropy and mean energy, so that 

Assumption 2: Uniform states 

Va,/3E R = E a =E p 

Va,/3S R = S a = Sp (115) 

This reduces the Generalised Landauer's principle to 
the form of: 

(AW) > -kT R AH]n2 
(AQ) > -kT R AHbx2 (116) 

where AH is the change in Shannon information over the 
course of the transformation. This is the usual form in 
which Landauer's principle is encountered. 

The necessary and sufficient conditions for these to 
hold is the weaker condition: 

Assumption 3: Uniform computing 

a (3 

Y,P(u)S a = ^P(/3)^ (117) 

a p 

B. Equilibrium Computing 

The simplifying assumption of uniform computing is 
made so universally, that it might be questioned whether 
there is any value to considering non-uniform computing. 
To answer this, consider what happens if the input and 
output states are constructed to be canonical thermal 
systems, at temperature T R) with the properties: 

Assumption 4: Equilibrium Computing 

E a -T R S a + kT R \nP(a) = C A 
E p -T R Sp + kT R \nP{P) = C A (118) 

where Ca is a constant, related to the overall size of the 
logical processing apparatus. This yields the relation- 
ships 

(AE) - T R AS = (119) 
and reduces the Generalised Landauer's principle to 

(AW) > (120) 

although 

(AQ) > -T R AS (121) 

still. The equality can, of course, only be reached in the 
limit of slow processes. 

The necessary and sufficient assumption for Equation 
mD]toholdis 
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Assumption 5: Zero mean work 

J2 a P(a) {E a - T R (S a + k In P(a))) 
= P(P) (E fJ - T R [Sp + fclnP(/3))) (122) 

Assumption [5] only implies the average work requirement 
can approach zero, over all the possible transitions be- 
tween logical states. Assumption @] ensures that there is 
a zero mean work requirement AW a ,p for all individual 
(a, 0) transitions. 

C. Adiabatic Computing 

To eliminate mean heat generation in the ideal limit, 
the necessary and sufficient condition is: 

Assumption 6: Zero mean heat genera- 
tion 

Y, P(a) (Sa + k InP(o)) = P (fi) ( S f> + k lnP(/3)) 

a (5 

(123) 

leading to 

(AQ) > (124) 

although this does not eliminate mean work requirements 

(AW) = (AE) (125) 

Again, this is only the expectation value over all transi- 
tions. To ensure that the mean heat generated AQ a ^ is 
zero for each individual (a, 0) transition, requires: 

Assumption 7: Adiabatic computing 

S a + klnP(a) = C B 

S f 3 + k\nP(f3) = C B (126) 

where Cb is an apparatus related constant. 

D. Adiabatic Equilibrium Computing 

Combining the assumptions of adiabatic and equilib- 
rium computing gives the requirement 

Assumption 8: Adiabatic equilibrium 
computing 

E a = Ep = Er 
S a + klnP(a) = C c 
S + k\nP(0) = C c (127) 



with equality being reachable as a limiting case, and Cc 
again a machine dependant constant. 

This result may seem surprising. It suggests that it is 
possible to design a computer to perform any combina- 
tion of logical operations, with no exchange of heat with 
the environment and requires no work to be performed 
upon it. This must be as true for logically irreversible 
operations as for logically reversible operations, and as 
true for logically indeterministic operations as for logi- 
cally deterministic operations. 

To understand this better, let us consider what hap- 
pens in adiabatic equilibrium computing. We can use the 
square well potential as the physical model of the logi- 
cal states, as the internal energy of these states is \kT . 
Varying the width of the square well potential for each 
input and output logical state satisfies the remaining con- 
ditions. 

Implementing the model of adiabatic equilibrium com- 
puting on the processes of Section IIIII simplifies the pro- 
cedure significantly: 

1. There is no need to resize the input states, as these 
will already be canonically distributed. Steps 1 
through to 3 are redundant. 

2. Potential barriers are inserted into the a states, cor- 
responding to the conditional probabilities P(/3\a), 
as in Step 4. 

3. The separate portions of the output states are 
brought into adjacent positions as in Step 5. 

4. The potential barriers within each output states 
are removed, as in Step 6. 

5. These output states are already canonically dis- 
tributed. There is therefore no need for a resizing 
of the output states and Step 7 through to 9 are 
unnecessary. 

None of these stages require any work to be performed 
upon the system or exchange of heat with the environ- 
ment. The computation is reduced to a process of rear- 
ranging a canonical ensemble from one set of canonically 
distributed orthogonal subensembles into a different set 
of canonically distributed orthogonal subensembles, in 
accordance with the computational probabilities P(/3\a). 

As the probabilities of the different output states can- 
not change between logical operations 14 then the canon- 
ically distributed output states can be used as canoni- 
cally distributed input states to any new logical opera- 
tion. This thermodynamic model may therefore proceed 
indefinitely without generating any heat or requiring any 
work. 



which yields, Vcc, 



AW a ,p > 

AQ Qi/3 > (128) 



By definition anything that changes the probabilities of a state 
must be a logical transformation of the data. 
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Before leaving this subject, let us just note one feature 
of equilibrium computing. Logically deterministic, irre- 
versible computations are able to avoid generating heat, 
in this model, by increasing the size of the physical states 
representing the logical states. This does not mean that 
the logical processing apparatus itself needs to be increas- 
ing in size. Although the size of the individual states 
has increased, the number of logical states has decreased 
(by the definition of a logically deterministic, irreversible 
computation!). Whenever the equality in Assumption [5] 
holds, the two effects cancel out and the overall size of 
the logical processing apparatus can remain constant. 



VI. THERMODYNAMIC REVERSIBILITY 

We have not yet examined the question of whether 
these operations are thermodynamically reversible. This 
is a subtle question and depends upon what one takes 
to be the statistical mechanical generalisation of thermo- 
dynamic entropy and thermodynamic reversibility. We 
will first discuss how this appears from the perspective 
of three different approaches to entropy, and then from 
a definition based on thermodynamic cycles that is not 
directly based upon any definition of entropy. 

It is worth remembering that a net increase in entropy 
is considered is taken as a sign of irreversibility because 
net decreases in entropy cannot occur (or are unlikely). 
An 'entropy' that can be systematically decreased may 
be a useful indicator of some properties, but its increase 
cannot automatically be regarded as an indicator of ir- 
reversibility, whether thermodynamic or of some other 
kind. 

We will consider three possible conditions for thermo- 
dynamic reversibility and irreversibility: 

1. The thermodynamic entropy is the entropy of the 
individual state. If the system is in logical state a, 
then the thermodynamic entropy is S a . The net 
entropy change for a particular logical transition, 
from logical state a to logical state f3 is: 



Tr 



(129) 



A transition is thermodynamically reversible if the 
decrease in individual state entropy from the input 
to output logical states is equal to the heat gener- 
ated in the heat bath, divided by the temperature 
of the heat bat. A transition is thermodynami- 
cally irreversible if the decrease in individual state 
entropy is less than this. Decreases in individual 
state entropy greater than this cannot occur. 

2. The thermodynamic entropy is the entropy of the 
individual state, but is only non-decreasing on av- 
erage. If the system is in logical state a, then the 
thermodynamic entropy is S a , but this may de- 
crease provided it does not decrease on average. 



The average change is: 



8 a R 



A logical transformation of information is thermo- 
dynamically reversible if the average decrease in in- 
dividual state entropy over all the transitions from 
input to output logical states is equal to the av- 
erage heat generated in the heat bath, divided by 
the temperature of the heat bath. The transforma- 
tion is thermodynamically irreversible if the aver- 
age decrease in individual state entropy is less than 
this. Average decreases in individual state entropy 
greater than this cannot occur. 



3. The thermodynamic entropy is the Gibbs-von 
Neumann entropy of the marginal statistical 
states. If the statistical state of the system is 
p = *}2 a P{ct)p a i the thermodynamic entropy is 
— fcTr [p In [p]] . A logical transformation of informa- 
tion is thermodynamically reversible if the decrease 
in Gibbs-von Neumann entropy from the input to 
output statistical states is equal to the average heat 
generated in the heat bath, divided by the temper- 
ature of the heat bath. The transformation is ther- 
modynamically irreversible if the decrease in Gibbs- 
von Neumann entropy is less than this. Decreases 
in Gibbs-von Neumann entropy greater than this 
cannot occur. 



The first two conditions imply thermodynamic irre- 
versibility for logically deterministic, irreversible opera- 
tions. Unfortunately, it will be shown neither condition 
can consistently account for logically indeterministic op- 
erations, which can systematically decrease the relevant 
entropy measure by quantities greater than should be 
permitted. 

The third condition gives an entropy that is consis- 
tently non-decreasing (provided there are no spontaneous 
or pre-existing correlations with heat baths). Logically 
indeterministic operations do not decrease this entropy. 
On the other hand, logically irreversible operations no 
longer necessarily increase this entropy measure either. 
According to the Gibbs-von Neumann measure, all logi- 
cal operations may be implemented in a thermodynami- 
cally reversible manner. 

We will be making the standard assumptions that all 
processes can take place with ideal heat baths and suffi- 
ciently slowly that equalities are reached as the limiting 
cases. Without these assumptions no process can be ther- 
modynamically reversible. We will therefore replace the 
appropriate inequalities with equalities. 
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A. Individual logical state entropy 

The net individual state entropy change, for a partic- 
ular logical transition, gives: 



S/3 — S a 



AQ a ^ 



> fcln [P{f3\a)\ 



(131) 



Allowed logically deterministic transitions require 
P(P\a) = 1. The equality is automatically reached for 
logically deterministic, reversible transitions, and which 
are therefore thermodynamically reversible. For logi- 
cally deterministic, irreversible transitions, the equality 
requires w a = 1. This is only possible if no other input 
logical states are allowed. Such an operation would be 
trivially logically reversible as there is only one permissi- 
ble input logical state. So according to this entropy mea- 
sure, logically deterministic irreversible transitions must 
be thermodynamically irreversible. 
As In [P(/3|a)] < it is possible that 



>/3 



S a 



AQ a J3 
T R 



< 



(132) 



This gives a net decrease in individual state entropy. For 
this to happen, the transition must be logically indeter- 
ministic. Optimally implemented, logically indeterminis- 
tic, reversible transitions will always decrease individual 
state entropy. 

If an entropy increase is indicative of thermodynamic 
irreversibility because entropy decreases are impossible, 
this measure of entropy cannot be seen as a good in- 
dicator of thermodynamic irreversibility. Any apparent 
irreversibility can actually be reversed. 



B. Average state entropy 



and the net mean change in individual state entropy of 
the system and environment is strictly increasing. Again 
logically deterministic, irreversible operations must be, 
on average, individual state entropy increasing. 

The problem with the argument should be immediately 
apparent: for logically reversible, indeterministic opera- 
tions AH > and by the same reasoning and arguments 
it is possible that 



AS NIB df = -kAH ln2 < 



(136) 



Not only can logically indeterministic operations reduce 
individual state entropy on individual transitions, they 
can even reduce this entropy on average. 



C. Gibbs-von Neumann entropy 

The entropy measure which includes the effects of the 
statistical mixture over the states, the Gibbs-von Neu- 
mann entropy over the ensemble, gives the initial entropy 
of the logical processing system: 



where 



Sj = -fcTrfp/ln^/] 



Pi = XI P ( a )P<* 



(137) 
(138) 



and the final entropy: 

S F = -kTr[p F la[p F ]} (139) 

where 



In statistical mechanics fluctuations occur. Perhaps 
the demand for a strictly non-decreasing entropy might 
be the problem. What of the average change in entropy? 
Does this give a good indicator of thermodynamic irre- 
versibility? 

This gives: 



Tr 



(133) 

What we have here is the ideal limit case of GLPA, with 
ASnibdf now representing the average change in en- 
tropy: 



{AQ a . p ) 
Tr 



(134) 



Logically deterministic, irreversible operations have 
AH < then 



AS 



NIBDF 



> 



(135) 



In [l3[ it is argued that the Gibbs-von Neumann en- 
tropy is indeed the correct statistical mechanical gener- 
alisation of thermodynamic entropy, although this iden- 
tification has not been assumed anywhere within this pa- 



ir, 



per 

When we consider the Gibbs-von Neumann entropy, 
the most appropriate form of the GLP is GLP3. In this 
case, the limiting behaviour gives 



AShb + AS = 



(141) 



As any logical operation may reach this limit, the Gibbs- 
von Neumann entropy regards all logical operations as 
being possible in a thermodynamically reversible manner. 



15 We have calculated the Gibbs-von Neumann entropies for indi- 
vidual states, in canonical distributions, but even here the calcu- 
lation of the mean work requirements and mean heat generated 
did not depend upon any identification of this as a thermody- 
namic entropy. 
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D. Discussion 

As some of these results may seem surprising or 
counter-intuitive, and appear to contradict widely stated 
expressions of the implications of the thermodynamics of 
logically irreversible operations, let us examine them in 
more detail. 



1. The RLE-LE cycle 

First, let us take the examples of the logically deter- 
ministic, irreversible Reset to Zero (RTZ) operation and 
the logically reversible, indeterministic Unset from Zero 
(UFZ) operation ( Appendix [A| . If the argument is ac- 
cepted that the optimal procedure to implement RTZ is 
entropy increasing, then it must also be accepted that the 
optimal procedure for U FZ can be entropy decreasing. 

That this must be the case can be seen by considering 
the Reverse Landauer Erasure (RLE) operation imme- 
diately followed by the Landauer Erasure (LE) opera- 
tion. If these two procedures are matched in terms of the 
probabilities, input and output states, then the result is 
to leave both the logical system and the environment in 
their initial states. The total entropy must be the same 
at the end of such a procedure, as at the start, and it 
follows if it increases during LE, then it must decrease 
during RLE. 

As a simple example, using the assumptions of uniform 
computing, and an initial input state of 0, the process of 
RLE extracts fcTln2 heat from the environment, and 
converts it into work. The output state of RLE is an 
equiprobable distribution of logical states and 1, each 
of which has the same entropy as the initial state. 

This is input to the LE procedure, which requires 
kT In 2 heat to be generated in the environment and 
leaves the output state as 0. The system and environ- 
ment are left in the same logical and thermodynamic 
states as at the beginning of the process. There is a 
zero net work requirement and a zero net heat genera- 
tion. The combination of RLE followed by LE is clearly 
a thermodynamically reversible cycle. 

It follows that the net change in entropy over the course 
of the two operations must be zero for both system and 
environment. To argue that the net change in entropy for 
the LE procedure is k In 2, requires, for the overall change 
in entropy to be zero, the change in entropy during the 
RLE operation to be — fcln2. 

Both the individual state entropies, and the average 
state entropy, do indeed decrease by A: In 2 during the 
RLE operation. The Gibbs-von Neumann entropy re- 
mains constant, as the mixing entropy increases by £Tn2 
to compensate. During the course of the LE operation, 
the individual state, and average state, entropies increase 
by A; In 2. In the conventional operation of the LE pro- 
cess, this is associated with heat generated in the envi- 
ronment, and is often considered to be the source of an 
irreversible entropy increase. However, we can clearly see 



that from the point of view of the Gibbs-von Neumann 
entropy, there is a compensating reduction of k In 2 asso- 
ciated with the reduction in the mixing entropy. 

2. Uniform Computing 

We can easily generalise this to situations where the 
quantity of information erased is less that 1 bit 16 , and 
in doing so will see more clearly the need to optimise 
the operation to the probability distribution. We simply 
need to implement an UFZ(p) operation, followed by an 
RTZ(p) operation. 

We start with a standard "atom in a box", and the 
partition divides the box exactly in half. The atom is on 
the left hand side (which represents logical state 0) with 
certainty. 

1. RLE(p). 

The RLE(p) operation consists of the following 
steps: 

(a) Isothermally move the partition to the right 
hand side, extracting W\ — kTh\2 heat as 
work. 

(b) Insert the partition at location x = pL in the 
box, where the width of the box is L. The 
atom is, with probability p, on the left hand 
side of the partition. 

(c) Isothermally move the partition to the centre 
of the box (x — \L). If the atom is on the left 
hand side, the work requirement is kT\n(2p) 
while if it is on the right, the work requirement 
is A:7Tn(2(l —p)). The mean work required in 
this stage is 

W 2 = kT (p \np + (1 - p) ln(l - p) + In 2) (142) 

so the net work for the operation is 

Wi + W 2 = kT(plnp+ (1 -p)ln(l - p)) (143) 

which is negative, representing a net extrac- 
tion of work. 

Now, we find that the individual state entropy, 
and average state entropy, remains the same as 
at the start of the operation, despite the fact 
that kT (p\np + (1 — p) ln(l — p)) work has been 
extracted from the heat bath. From the point of 
view of the Gibbs-von Neumann entropy, this is 
compensated by the increase in mixing entropy be- 
tween the two logical states. 

2. LE(p). 

If we follow this with an LE(p), we have the steps: 



This cycle was detailed in 
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(a) Isothermally move the partition to the posi- 
tion x — pL. If the atom is on the left hand 
side, the work requirement is — fc7Tn(2p) while 
if it is on the right, the work requirement is 
— fcTln(2(l —p))- The mean work required in 
this stage is 

W 3 = -kT(p\np + (1 -p)ln(l -p)+ln2) (144) 

(b) Remove the partition from the box. 

(c) Insert the partition in the right hand side of 
the box and isothermally move it to the centre. 
This requires W\ = fcTln2 work, so the net 
work is 

W3 + W4 = -kT(plnp+ (1 -p)ln(l - p)) (145) 

Again, both the individual and average state en- 
tropy are unchanged, while work is converted to 
heat in the environment. The Gibbs-von Neumann 
entropy, however, shows a compensating decrease 
in mixing entropy. 

The net work and net heat generated, over the 
course of the cycle, is zero: 



Wi + W 2 + W 3 + W4, = 



(146) 



If the heat generated in the environment during 
the LE(j>) operation is an indicator of an irre- 
versible entropy increase, we have to explain a cor- 
responding systematic reduction in entropy dur- 
ing the RLE(p) operation. As we noted, en- 
tropy increases are associated with irreversibility 
precisely because corresponding systematic entropy 
decreases are supposed to be impossible. 

3. LE{p'). 

Let us now consider following the RLE(p) opera- 
tion with LE(p'), where the erasure operation has 
been optimised for a different probability distribu- 
tion. 

(a) Isothermally move the partition to the posi- 
tion x = p'L. If the atom is on the left hand 
side, the work requirement is — fcTln(2p') 
while if it is on the right, the work require- 
ment is — /cTln(2(l — p')). The mean work 
required in this stage is 

W 5 = -kT (plnp' + (1 - p) ln(l - p') + In 2) (147) 

(b) Remove the partition from the box. 

(c) Insert the partition in the right hand side of 
the box and isothermally move it to the centre. 
This requires We = kT In 2 work, so the net 
work is 



The net work required over the RLE(p)-LE(p') cy- 
cle is 



Wi -I 
+W 5 



w 2 

-W 6 



= kT I p In 



(149) 

with equality occurring if, and only if, p = p' . 

Once again, both the individual and average state 
entropy are unchanged. In this case, however, the 
cycle generates a net heat in the environment, un- 
less p = p' . This cycle is, in general, thermody- 
namically irreversible. 

From the point of view of the Gibbs-von Neumann en- 
tropy, it is the removal of the partition from the location 
x = p' ' L, when the probability is p, that associated with 
an uncompensated entropy increase. We can see this by 
noting that if we reinsert the partition at x = p' L, we do 
not recover the previous statistical state, as the proba- 
bility of the atom being on the left hand side would then 
be p'. To recover the statistical state we need to rein- 
sert the partition at x = pL, then move it isothermally 
to x — p'L. This isothermal movement of the partition 
requires, on average: 



fpln 


~P~ 




y. 



+ (l-p) In 



l-p 



l-p' 



> 



W 5 + W 6 



-kT(p\np' + (l-p)hi(l-p')) ( 148 ) 



work to be performed. 

Even so, let us note that had LE(p') in fact followed 
the RLE(p') operation, it would have been thermody- 
namically reversible. The physical process involved in 
performing the LE(p) (or LE(p')) operation cannot be 
said to be intrinsically thermodynamically reversible (or 
irreversible) in itself. Whether it is thermodynamically 
reversible, or not, depends upon the statistical state upon 
which it acts. 



3. Adiabatic Equilibrium Computing 

Let us look at the same logical cycle, but with a differ- 
ent computing model: adiabatic equilibrium. Again we 
start with a standard "atom in a box" . As the atom is in 
logical state with certainty, the conditions of Equation 
11271 require that logical state occupies the entire box. 

1. RLE{p). 

The RLE(p) operation now consists of the single 
step: 

(a) Insert the partition at location x = pL in the 
box, where the width of the box is L. The 
atom is, with probability p, on the left hand 
side of the partition. 

No work is required or heat generated. The 
individual and average state entropies have de- 
creased, with the average state entropy decreas- 
ing by k (plnp + (1 — p) ln(l — p)). The Gibbs-von 
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Neumann entropy remains the same, as the mixing 
entropy compensates for this. 

2. LE(p). 

If we follow this with an LE(p), we have the step: 

(a) Remove the partition from the box. 

Both the individual and average state entropy are 
increased, with the average state entropy increas- 
ing by k [p\np + (1 — p) ln(l — p)). The Gibbs-von 
Neumann entropy, however, shows a compensating 
decrease in mixing entropy. 

We see how, in the case of adiabatic equilibrium 
computing, the generation of heat in the environ- 
ment is replaced by changes in the entropies of the 
individual states (or, as [H refers to it, the non- 
information bearing degrees of freedom of the ap- 
paratus). Although there is an increase in such 
entropies during the LE(p) process, there is an ex- 
actly equivalent decrease during the RLE(p) pro- 
cess. Again, if we take the increase during LE(p) 
to be indicative of a thermodynamic irreversibility, 
we are left with the challenge of accounting for the 
systematic decrease during the RLE(p) operation. 

3. LE(p') 

Following RLE(p) with an LE(p') operation un- 
der the assumptions of adiabatic equilibrium does 
not entirely make sense, as adiabatic equilibrium 
requires the physical representation of the logical 
states is tailored to the probability of the state 
occurring. However, we may consider the opti- 
mum implementation of RTZ(p'), on the assump- 
tion that the probability of the logical state is p\ 
with the partition initially located at x = pL and 
the process leaving the system in a state compatible 
with adiabatic equilibrium computation. 

(a) Isothermally move the partition to the posi- 
tion x = p' L. If the atom is on the left hand 

side, the work requirement is — /cTln^^L^ 

while if it is on the right, the work requirement 

is — fcTln (yjf-^j- The mean work required 17 

in this stage is 
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1 -p' 

with equality occurring if, and only if, p = p' . 



17 Note that, had the probability of logical state actually been p' , 
the work required would have been: 
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+ (l-p') ln 
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1-p' 

so work would have been extracted in the process. 
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(b) Remove the partition from the box. 

The net work required over the RLE(p)-LE(p') cy- 
cle is again 



(phi 






y. 



+ (l-p) In 
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1-p' 
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Once again, from the point of view of the Gibbs-von Neu- 
mann entropy, it is the removal of the partition from the 
location x — p'L, when the probability is p, that is asso- 
ciated with an uncompensated entropy increase. 



4- Generic logical operations 

Now let us consider a generic logical transformation 
of information. Start with input logical states a, phys- 
ically represented by states with energies and entropies 
E a and S a , and define a logical operation by the tran- 
sition probabilities P((3\a) to the output logical states (3 
with physical state energies and entropies Ep and Sp. 

To thermodynamically optimise the physical process, 
we need a probability distribution P{a). The (3 output 
states will then occur with probabilities 
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Writing 



Pi 



then the optimal thermodynamic cost of this is: 
(AW) = (Tr [Hpi] - T R S [ Pi ]) - (Tr [Hp F ] - T R S [p F ]) 



+kT R J2P(a,P) In 
(AQ) = -T R S[ PI ]+T R S[ PF ] 



PW 



P(a) 



(154) 



We can now define a physical process, that acts upon 
the physical states {/3}, and evolves them into the phys- 
ical states {a}, with probabilities 18 given by 



P(/3\a)P(a) 
E a P((3\a)P(a) 



(155) 



18 For logical operations taking as input states {/?} and producing 
output states {«}, we will use the notation II for the correspond- 
ing probabilities. 
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It is straightforward to see that if this acts upon states 
{/3}, occurring with probabilities P((3), then it produces 
the states {a} with probabilities P{a). If the physical 
process is optimised for these probabilities, then the ther- 
modynamic cost is 



(Alf n ) = - (AW) 
(AQu) = - (AQ) 



(156) 



So for any logical transformation of information, opti- 
mally implemented, there exist a second operation, which 
when optimally implemented restores the original statis- 
tical state, and for which the total expectation value of 
the work requirement, and the total expectation value of 
the energy generated in the environment, is zero. This is 
true regardless of whether the original operation is logical 
reversible, irreversible, deterministic or indeterministic. 

As we have noted before, however, to achieve this op- 
timum for logically irreversible operations, the physical 
process must take into account the probability distribu- 
tion P(a) over the input logical states. One cannot cre- 
ate a physical process, that implements a logically ir- 
reversible operation, which will be thermodynamically 
optimal for every probability distribution over the input 
logical states. This differs from logically reversible oper- 
ations, which may be represented by a physical process 
which is thermodynamically optimal for any probability 
distribution over the input logical states. 

We will now look at the effect of an operation that is 
not optimised for the right set of probabilities. Suppose 
we have an operation with the same transition probabili- 
ties II(a|/3), above, but the physical process has been op- 
timised for the input probability distribution IT(/3). The 
output states are expected to occur with probabilities 



H(a 



= E n (< 





|/3)n(/3) 
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and the expected thermodynamic cost, from Equation 
[TBI is: 

(AW U ) = E 1%) (E a - T R S a ) - J2 n (/3) ( e p - t rSp) 

a 



-kT R n(a, 13) In 

a,0 



Wp 



which can be rearranged to give 
(AWd)+(AW) = kT R Y,P{a,P) ^ 



IL(/3\a)P(a) 



> 
(161) 

where IL{P\a)IL(a) = ll{a\0)U(j3). 

Equality can occur in two ways. Firstly, and most 
simply, if U(j3) = P((i). The input states to the IT(a|/3) 
operation occur with the optimal probabilities. 

Secondly, if the second operation is a logically re- 
versible operation, then 



V/3 [U(a\f3) ^O^VaVa n(a'|/3) = 0] 



(162) 



As n(a|/9) = P(a\f3), it follows the first operation must 
have been logically deterministic: 



V/3 [P(a\f3) ^O^VaVa P{a'\ 
Together this means 



0] (163) 



P(a[ 



Wp 
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and (AWn) + (AW) = 0, regardless of the values of wp. 
This shows, once more, that logically reversible opera- 
tions may be thermodynamically optimised without ref- 
erence to the probability distribution over their input 
states. 

A corollary to this is worth noting. While the second 
logical operation, if logically reversible, may be imple- 
mented and optimised without reference to the probabil- 
ity distribution over the input states, its very definition 
depends upon the probability distribution over the input 
states of the first operation. The first operation is defined 
by the set of transition probabilities {P(/3\a)}, while the 
second is defined by 



P(a\, 



P(P\u)P(a) 
£ a P(/3|a)P(a) 



(165) 



(158) 



with w a = n(a) wp = n(/3) and U(a, (3) = II(a|/3)II(/?). 

The input states do not occur with II(/3) but with 
P(P)- The actual thermodynamic cost incurred is: 



-kT R J2P(a,f3) In 

a,P 



Wp 
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The combined cycle now has a cost 
(AWjj) + (AW) = kT R P(at, 13) In 

a,p 



w a P(P) 
wpP(a) 



(160) 



There is, in general, only one way to make this inde- 
pendant of {P(a)}: if the first operation is logically re- 
versible, then P(a\(3) 6 {0,1}. The second operation is 
now logically deterministic and II(a|/3) s {0,1} does not 
require the {P(a)}. 

We can summarise this, as follows: if an operation, 
{P((3\a)} is logically reversible, then it is possible to 
calculate a (logically deterministic) reverse operation, 
{II(q;|/?)}, independantly of the first input probability 
distribution, {P(a)}. However, if {P(/3|a)} is logically 
indeterministic, then optimising the reverse operation re- 
quires the output probability distribution {P (/?)}. 

Conversely, if an operation {P(/3\a)} is logically deter- 
ministic, then it is possible to thermodynamically opti- 
mise a (logically reversible) reverse operation {±l(a|/3)}, 
independantly of the first output probability distribution 
{P(f3)}. However, if {P((3\a)} is logically irreversible, 
then the very calculation of the probabilities {n(a|/3)} 
require the first input probability distribution {P(a)}. 
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In general, it is only for logically deterministic, re- 
versible operations (which are permutations) that one 
can construct optimal reverse operations independantly 
of the probability distributions. 

E. Thermodynamic irreversibility 

The reverse operations considered in the preceding dis- 
cussion have the property of restoring the original statis- 
tical state of the logical system. They do not, in general, 
restore the original logical state. The question of what is 
the 'correct' thermodynamic entropy to use in such sit- 
uations is not uncontroversial and can depend upon dif- 
fering physical interpretations of the probabilities of the 
initial and final logical states. It will therefore be helpful 
to consider an approach to thermodynamic reversibility 
which does not depend upon such definitions. 

We will use this to discuss further that the thermody- 
namic optimisation of logically irreversible operations is 
not possible without specifying the probability distribu- 
tion over the input states. Then we consider two addi- 
tional sources of thermodynamic irreversibility that occur 
in the practical construction of information processing 
systems. 

1. Thermodynamic cycles 

In phcnomcnological thermodynamics, in any closed 
cycle, where a system returns to its initial state, the total 
heat generated in heat baths in the process, must satisfy 

Ef^° ( 166 ) 

i 

As is well known, in statistical mechanics this can no 
longer be relied upon. There is some probability for the 
equality being violated. However, provided the system 
does return to its initial macroscopic state with certainty, 
then 

£ M > (167) 

still holds. We will regard such a cycle for which the 
equality holds, to be a thermodynamically reversible cy- 
cle, and use the following definition 19 of a thermodynam- 
ically reversible process: 

If a given physical process can, in principle, 
be included in at least one thermodynami- 
cally reversible cycle, then it is a thermody- 
namically reversible process. 



We define the condition in this way to take into account the 
fact that for any physical process, it is always trivially possible 
to find some closed cycle incorporating that process for which 
inequality is strictly positive. 



To say otherwise would require one cither to say that the 
overall cycle is thermodynamically reversible, although 
one of the steps in the cycle is not (which challenges what 
it could possibly mean to refer to that step as thermo- 
dynamically irreversible) or to say that the overall cycle 
is thermodynamically irreversible, despite the fact that 
it restores the original state with certainty and generates 
no net heat in any heat bath (and which means that the 
entropy of the universe must be the same at the end as 
the start of the cycle) . 
Conversely 

If a given physical process cannot, even in 
principle, be included in any thermodynami- 
cally reversible cycles, then it is a thermody- 
namically irreversible process. 

To avoid interpretational problems over probability, we 
will require that the thermodynamically reversible cycle 
starts, and ends, with the system in a physical state that 
represents a fixed logical state a, with certainty. 



2. Optimal Implementations 

Take any logical operation, defined by the set 
{P([3\a)}, and construct a physical implementation of 
that operation, optimised for the values w a and wp = 
J2 a P(/3\a)w a . This physical process will implement the 
{P(f3\a)} operation regardless of the input state proba- 
bilities. 

We now also construct two further operations: a log- 
ically reversible, indeterministic operation, generalising 
the UFZ operation, that acts on a as the sole possible 
logical input state, and outputs state a with probability 
P{a) = w a ; and a logically irreversible, deterministic op- 
eration, generalising RTZ, that acts on the logical states 
{/3} , and always outputs logical state a. The physical 
implementation of this second operation is optimised for 
probabilities P(f3) — wp. Both these are well defined 
physical processes. 

It is clear that the sequence of these three operations 
forms a closed cycle, starting and ending in logical state 
a, with certainty. It is trivial to show that the optimal 
implementation of these operations produces a net ther- 
modynamic cost of zero, over the course of the cycle. The 
cycle is, unquestionably, a thermodynamically reversible 
cycle. The given physical process that implements the 
logical operation must, then, be regarded as a thermody- 
namically reversible process. 



3. Suboptimal Implementations 

If we had used a different initial operation, gener- 
ating the logical state a with probability P'(a), and 
a final operation optimised for probabilities P'{(3) = 
J2 a P(P\ a )P'i a )> then it is straightforward to show the 
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cost would be 



W = kT^P'{a,(3) In 

a,0 



P'(a)wp 
P'{[i)w a 



> 



(168) 



Equality is in general reachable if either w a — P'{a) or 
Va, (3, P(P\a)w a G (0, wp), the latter being possible only 
if the logical operation {P((3\a}} is logically reversible. 
This leaves us with the following conclusions: 

1. For any logical operation {P((3\a)}, there exist 
physical implementations of that operation which 
can be included in thermodynamically reversible 
cycles. 

2. For any logical operation {P((3\a)}, and for any 
given probability distribution P(ct) over the input 
logical states, there exist physical implementations 
of that operation which can be included in thermo- 
dynamically reversible cycles. 

3. A given physical implementation of logical opera- 
tion {P(/3\a)} cannot be included in thermodynam- 
ically reversible cycles for generic probability distri- 
butions over the input states unless it is a logically 
reversible operation. 

It is not possible to characterise a particular physical 
process, that implements a logically irreversible opera- 
tion, as thermodynamically reversible, indcpcndantly of 
the specification of the statistical state on which it acts. 
Does this mean that we cannot characterise the physical 
process as thermodynamically reversible, at all? 

This situation is not unknown in statistical mechanics, 
or even phenomenological thermodynamics. Let us con- 
sider a large container, divided in half by a removable 
partition, and in the container is a macroscopic gas. The 
pressure on both sides of the partition is initially equal, 
and the gas is always kept in isothermal contact with a 
single heat bath. Removing and reinserting the partition 
is clearly thermodynamically reversible. 

If we slowly, isothermally, slide the partition to the left, 
compressing half the gas and expanding the other half, 
until the compressed gas occupies only one-third the con- 
tainer, the pressure on the left side is double the pressure 
on the right side (net work is required). Removing and 
reinserting the partition at this off-centre position is not 
thermodynamically reversible. 

This thermodynamic irreversibility is not simply due 
to the off-centre position of partition. Start with the par- 
tition in the centre, but now with gas initially prepared 
to be at twice the pressure on the right hand side of 
the container as on the left hand side. Simply removing 
the partition from the centre of the box is now thermody- 
namically irreversible. Isothermally moving the partition 
to the left until the left hand side holds only one third 
of the container's volume equalises the pressure (and ex- 
tracts work) . Now the off-centre removal and reinsertion 
of the partition becomes thermodynamically reversible. 



The parallel to the model used for logical operations 
should be clear 20 . A given sequence of actions cannot, in 
general, be regarded as thermodynamically reversible in- 
dependantly of the state on which they act. To describe 
a phenomenological thermodynamic process as thermo- 
dynamically reversible it is necessary to specify both the 
sequence of actions and the state on which they act in the 
definition of the physical process. This carries over into 
statistical mechanics and, as we have seen above, into the 
thermodynamics of computation. 

The situation also bears some similarity to data com- 
pression from a signal source. A given coding scheme 
will only be optimal for a particular distribution of prob- 
abilities of signals from the source. Should the signals, 
in fact, be generated with a different probability distri- 
bution, then the mean length of the encoded signals will 
be greater than the Shannon information of the source. 
That Shannon's coding theorem is of practical utility in- 
dicates that it is not inconceivable that there may be 
information processing problems where the probability 
distribution over the logical states may be available when 
designing optimal physical implementations. 



4- Uncertain Operations 

If the logical operation acts upon a set of statistical 
states, but it is uncertain which operations have acted 
upon the system in the past, an additional source of ther- 
modynamic irreversibility may occur. As an example of 
this, let us consider a bit that has been deterministically 
set to either zero or one, from a standard state a, and 
now needs to be reset to the standard state. 

If the first operation set the bit to zero, the operation 
is UFZ(1), and the work required was 

AW = (E - T R S ) - (E a - T R S a ) (169) 

and if set to one, UFZ(0) gives 

AW 1 - (E 1 - TflSi) - (E a - T R S a ) (170) 

If the reset operation is optimised with values wo + u>\ = 
1, then it is RTZ(w ), 

AW R0 = (E a -T R S a )-(E -T R So)-kT R lnw 
AW R1 = {E a -T R S a )-{E l -T R S l )-kT R \nw l 

(171) 



giving total costs 



AW T0 = AW R0 + AW 
AW T1 = AW R1 + AWi 



-kT R hnwo > 
-kT R \n Wl > (172) 



20 Indeed, if we are considering a statistical mechanical N-atom gas, 
with N=l, it is exactly the same model. 
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The equalities can be reached by setting wq = 1 or wi — 
1, respectively, but this is only possible if the other is 
zero - which would require an infinite amount of work if 
the wrong operation had taken place! 

If we assign non-zero probabilities to the set operations 
of pa and p\ , then the expected cost for the cycle is 

AW T = -kT R Pilv-Wi > -kT R ^p l \np l > 

i=0,l i 

(173) 

with the equality occurring if Wi = Pi- Clearly this 
is a thermodynamically irreversible cycle, despite the 
fact that each of the three logical operations (UFZ(1), 
UFZ(0), RTZ(wo j) can be individually incorporated in 
a thermodynamically reversible cycle. What is the source 
of the irreversibility? 

There are a number of ways one can regard this. Both 
the deterministic set operations are, in themselves, ther- 
modynamically reversible. It could be argued that the 
irreversibility in whichever of the AWto or the AWri 
cycles that actually took place, is then through the reset 
operation, which was designed for the possibility of either 
deterministic set operation. 

A different way to perceive the situation is to regard 
the situation as either being AWto, which may be ther- 
modynamically optimised by setting wo = 1, or AWri 
which may be optimised by wi — 1. In either case the 
cycle becomes thermodynamically reversible. The source 
of thermodynamic irreversibility would then be that the 
reset operation was not optimised for the correct prob- 
abilities (which must now be regarded as either po = 1 
or pi = 1, corresponding to which operation actually did 
take place). 

Yet another way would be to consider a new class of 
operation: an 'uncertain' operation, where there is an un- 
certainty as to which actual operation took place. In this 
case we have an 'Uncertain Set' operation, which could 
be defined as p UFZ(1) + p x UFZ(0). This operation 
has a work requirement: 

AW V = ]T Pi (Ei - TnSt) - (E a - T R S a ) (174) 

i=0,l 

Viewed as a logical operation, this would take as input 
logical state with probability one, and output states 
and 1 with probabilities P q and p\ . The optimal imple- 
mentation of such a logical transformation of information 
would be UFZ(p ), which has cost 

AW^Y, P l (E l -T R S i )-(E a -T R S a ) + kT R Y J P l ^p t 

i=0.1 i 

(175) 

As a logical transformation of information, the 'Uncer- 
tain Set' operation is clearly sub-optimal. It is thermo- 
dynamically irreversible, as it cannot be included in any 
thermodynamically reversible cycle. 

What is the 'correct' way to view this? We are not sure 
this is a well-posed question. However, what all three 



explanations have in common is that the thermodynamic 
irreversibility is a consequence of the uncertainty over 
which logical operation took place. It is this that prevents 
the construction of a thermodynamically reversible cycle. 

Suppose we have a number of different process, la- 
beled with 7, and each implements a logical operation 
{P(f3\a, 7)}, optimised for input state probabilities P(a). 
The optimal cost for operation 7 is 

AW, = Yl P ^ - T * S P + kT * ln P ^M) 



-J2 P ( a ) ( E « - T R S - + kT R \nP{a)im) 

a 

where P(/3| 7 ) = £ a 7 )P(«) 

We now assign a probability P(j) to each logical 
operation occurring (and take for granted P(a, 7) = 
P(a)P(-f)). The cost of this 'Generic Uncertain Oper- 
ation' is 

{AW,) = Y P{a,M){E p -E a 

-T R [s p -S a -k\n^-^ (177) 

This produces the output states {/3} with probabilities 
P(P)=E ar/ P(P\a,l)P(a)P(j). 

Now to complete the cycle, we consider an optimised 
Reset operation, the acts upon states {(3} to produce 
the standard state a, and an optimal operation that acts 
upon a and produces the logical states {a} with proba- 
bility P(a). Combining these two has the cost 

AW R = Y, P (. a )( E <*-T R S a + kT R \nP{a)) 

a 

- Y P{P) (Ep - T R S P + kT R ln P(^78) 



giving a total cost for the cycle of 

{AW,) + AW R = kT R ]T P(J}, 7) In p^p^ > 

(179) 

Equality is reached only if P(/3,-f) = P(f3)P(j), i.e. 
there is no correlation between the occurrence of the 
(3 output states and which 7 operation actually took 
place. The thermodynamic irreversibility that occurs if 
P{ft\l) 7^ P{P) does not depend upon whether the oper- 
ation required to restore the original statistical state is 
logically reversible or logically irreversible. 

In the familiar case of the 'Uncertain Set '-Reset cy- 
cle there is a compression of the logical state space dur- 
ing the reset operation and the compensating increase in 
the non-information bearing degrees of freedom of sys- 
tem or environment may give the impression that the 
source of the thermodynamic irreversibility is the logi- 
cal irreversibility of the Reset operation. The 'Generic 
Uncertain Operation' shows this is not the case. In fact 
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an optimal operation that restores the P(a) distribution 
from the P(/3) distribution could be logically reversible 
and the cycle still be thermodynamically irreversible pro- 
vided P(/3\j) P(/3)- It is the uncertainty over which 7 
operation took place that is the source of the thermody- 
namic irreversibility. 

As before, this situation has well known parallels 
in standard statistical mechanics. The spread of gas 
molecules into a box, shielded from any outside inter- 
ference, can in principle be reversed. (Spin-echo exper- 
iments have even demonstrated similar reversals to this 
in the laboratory.) However, this reversal is very sensi- 
tive to uncertainty in the outside forces that act upon 
the gas. In a famous calculation, Borel showed that the 
gravitational influence of remote stars could change the 
microscopic state of an expanding macroscopic gas within 
seconds. Reversing that expansion would be possible, in 
principle, if there was highly detailed knowledge of the 
gravitational influence of the remote bodies on the gas (or 
if microscopic state of the expanded gas molecules turned 
out to be independant of that influence) but becomes im- 
possible when the gravitational influence is uncertain. 



5. Partial Operations 

A third reason for the occurrence of thermodynamic 
irreversibility is that the physical implementation of the 
logical operation is not able to take into account the ex- 
istence of correlations between systems, and can only act 
upon part of the total logical state 21 . We will show that, 
in this case, logically reversible operations are able to 
avoid the thermodynamic irreversibility, although logi- 
cally irreversible operations are still not always thermo- 
dynamically irreversible. 

Suppose the input logical states factorise into the prod- 
uct of two subsystems, with the logical states of the first 
system in the set {a} and the second system in {7}, so 
the joint system is described by the logical states {(a, 7)}. 
Now consider a logical operation that acts only on the a 
states, with probabilities P(f3\a). If the physical imple- 
mentation of this logical operation has no access to the 
7 system, then the physical implementation can only be 
optimised with respect to the marginal probabilities 



P(a)=5>(a, 7 ) 



(180) 



The system ends up in output states from the product 
of the states of the {7} and systems, {(/3,7)}, with 
probabilities 



(181) 



The resulting thermodynamic cost of the partially op- 
timised operation is: 

AW P = Y,P(fl(E p -T H (Sp-khiP(0))) 



- J2 p (a) (E a - T R (S a - k In P(a))()182) 

a 

where P(/3) = P{P-> 7) an d we have assumed 
Ep,i — Efj + E 1 

Sj3,i = Sp + S* 7 (183) 

An optimal operation for restoring the states (a, 7), 
with probabilities P(a, 7) has a thermodynamic cost of 

AWr = ^P(a,7)(£ Q -T fl (S Q -fclnP( a ,7))) 

- J2 7) (E - T R (S p - k In P(P, im 

0,1 



so the net cost for the cycle is 
AW R + AW P 



kT R 



P(a) 



£p(/3,7) m ^M| ,;i,sr,) 



0,1 



p(/?) 



This can be expressed as changes in conditional or cor- 
relation information: 



AWr + AW P 



= - J2 P(a,p,~/) (In P(j\0)- In P(j\a)) 



J2 P(<*,P,i) (in 



P(/3)P( 7 ) 



In 



P(a, 7 ) 



P(a)P( 7 ) 
Using the identity 

P(a, j\f3)P(/3\a) = P(/3, -y\a)P(a\0) (187) 
gives the form of conditional correlations: 
AWr + AW P _ Ej>M 7) A P(M\a) 



(186) 



kT R 



In ■ 



21 See also M 



P(a|/?)P( 7 |/3) 
As P(/3|a,7) = P(/3|a), then 

P( 7 ,/3|a) = P(/3|a, 7 )P(7|a) 
= P(f3\a)P( 7 \a) 



P(f3\a)P(~y\a) 
(188) 



(189) 
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The a states screen off any correlation between the j3 and 
7 states and the first term is zero, so: 



AW R + AW P 



]T P(a,/3, 7 )ln 



P{a\0)P(y\0) 



> 



( 19 °) 

Equality occurs if, and only if, f3 screens off any correla- 
tions between a and 7: 



P(a, 7 \f3) = P(a\(3)P( 7 \0) 



(191) 



This can happen directly if there is no initial correla- 
tion between the a and 7 systems, so that P(a, 7) = 
P(a)P{j). With P(P\a,i) = P((3\a) it follows 
P(a, p, 7) = P(a, /3)P(i) and from that Equation UM\ 
holds, as might be expected. 

To see the effect of logical reversibility, rewrite Equa- 
tion nnni as 



AW R + AWp 



P(a,P,l) (lnP(a\ ll p)-\nP(a\P)) 

(192) 

If the operation is logically reversible P(a\/3) £ {0, 1}. 
This gives 



P(a\/3) = 
P(a\0) = 1 



F(a,/3, 7 ) = 
P(a\y,0) = l 



(193) 



and the summation is identically zero. Logically re- 
versible operations avoid the thermodynamically irre- 
versible cost 22 . 



VII. CONCLUSIONS 

The focus on the process of Landauer Erasure can give 
the impression that Landauer's principle should be exclu- 
sively about the thermodynamics of logically irreversible 
processes and further that the heat generation of such 
processes implies thermodynamic irreversibility: 

To erase a bit of information in an environ- 
ment at temperature T requires dissipation of 
energy > fcTln 2. [H, H| 

in erasing one bit ... of information one dis- 
sipates, on average, at least fc_BTln(2) of en- 
ergy into the environment. [25j 

a logically irreversible operation must be im- 
plemented by a physically irreversible device, 
which dissipates heat into the environment 

M 



22 However, one should note that it is still possible for some logi- 
cally irreversible operation to satisfy the conditions for thermo- 
dynamic reversibility, Equation 11911 for particular correlations 
between the a and 7 systems. 



erasure of one bit of information increases the 
entropy of the environment by at least A: In 2 

na[pg27] 

any logically irreversible manipulation of data 
. . . must be accompanied by a correspond- 
ing entropy increase in the non-information 
bearing degrees of freedom of the informa- 
tion processing apparatus or its environment. 
Conversely, it is generally accepted that any 
logically reversible transformation of informa- 
tion can in principle be accomplished by an 
appropriate physical mechanism operating in 
a thermodynamically reversible fashion. [l| 

though it should be noted that not all advocates of 
Landauer's principle regard the process of erasure as nec- 
essarily thermodynamically irreversible: 

a logically irreversible operation . . . may be 
thermodynamically reversible or not depend- 
ing on the data to which it is applied. If it 
is applied to random data ... it is thermody- 
namically reversible, because it decreases the 
entropy of the data while increasing the en- 
tropy of the environment by the same amount 



In it was argued that there exists a valid thermo- 
dynamically reverse process to Landauer Erasure, but 
which needs to be classified as logically indeterministic, 
which we called Reverse Landauer Erasure (or RLE). 
Consideration of the thermodynamic consequences of the 
existence of this process led us to conclude there was no 
convincing evidence that logically irreversible operations 
had special thermodynamic characteristics. Instead, we 
hypothesised that a generalised form of Landauer's prin- 
ciple should be possible that made no reference to irre- 
versibility, whether logical or thermodynamic. This was 
expressed in two conjectures: 

(E) : Any logically irreversible transformation 
of information can in principle be accom- 
plished by an appropriate physical mecha- 
nism operating in a thermodynamically re- 
versible fashion. 

(F) : A logical operation needs to generate 
heat equal to at least —fcTln 2 times the 
change in the total quantity of Shannon in- 
formation over the operation, or: 



AW > kT\n2[Hi - H f 



l[pg. 362] 



In this paper we have both proved and generalised 
these conjectures. Our approach has been to take the 
widest definition of logical operations available and most 
general procedure for physically implementing these op- 
erations that we can. This requires us to consider logi- 
cally indeterministic operations as well as deterministic 
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ones, logically reversible operations as well as irreversible 
ones. 

Other papers have made some consideration of 
Landauer Erasure in the context of non-uniform 
temperatures (28|, entropy [2H [3Cl|. and energy |25|, while 
[3TI ] combines varying entropy and energy. Non-uniform 
input probabilities are considered in the proofs of [25L l29j| . 
The thermodynamics of logically indeterministic opera- 
tions does not seem to be considered before 0, Q, al- 
though [32] [Chapter VI] is close, and it is noticeable that 
[l[ refers throughout to deterministic computation. Dur- 
ing the preparation of this paper, a paper has appeared 
by Turgut p} deriving similar results using classical phase 
space arguments. 

General proofs of Landauer's principle seem hard to 
come by (as pointed out in [5j) although [251 ] derives sim- 
ilar results to those of Section Ull Ei but restricted to the 
RTZ operation, and under the assumption that logical 
states are represented by pure quantum states (an as- 
sumption shared with [33|). Here we allow logical states 
to be represented by density matrices and consider any 
logical operation. We have considered the most general 
setting for physically implementing classical logical oper- 
ations, covering and extending these earlier results. We 
derive the most general statement of Landauer's prin- 
ciple, prove it cannot be exceeded and give a limiting 
process which can achieve it. 

The general statement of Landauer's principle we ar- 
rived at is: 

Generalised Landauer's principle 

A physical implementation of a logical trans- 
formation of information has minimal expec- 
tation value of the work requirement given 
by: 

(AW) > (AE) - TAS (194) 

where (AE) is the change in the mean inter- 
nal energy of the information processing sys- 
tem, AS the change in the Gibbs-von Neu- 
mann entropy of that system and T is the 
temperature of the heat bath into which any 
heat is absorbed. 

The equality is reachable, in principle, by 
any logical operation, and if the equality is 
reached the physical implementation is ther- 
modynamically reversible. 

We have then shown how various additional assump- 
tions and simplifications can lead to more familiar ver- 
sions of Landauer's principle that can be found in the 
literature and these are special cases of the GLP. Gen- 
eralisations about the relationship between information 
processing and thermodynamic entropy based upon these 
special cases can be misleading. 

In particular, we have argued, counter to a widespread 
version of Landauer's principle, that there is nothing 



in principle, that prevents a logically irreversible oper- 
ation from being implemented in a thermodynamically 
reversible manner. What differs between logically irre- 
versible operations and logically reversible operations is 
that to thermodynamically optimise physical implemen- 
tations of the former it is necessary to take into account 
the probability distribution over the complete set of input 
logical states. A physical implementation of a logically 
irreversible operation, optimised for a particular input 
probability distribution, will not be thermodynamically 
irreversible for a different input probability distribution. 
If the physical implementation cannot access a correlated 
system, then logically irreversible operations may incur 
additional costs. 

As the practical business of actually building physi- 
cal devices to implement logical operations will typically 
not be able to make such optimisations, it is natural 
to assume an equiprobable distribution over a subsys- 
tem, and expect thermodynamic irreversibility. Never- 
theless the point remains: in principle it is always possi- 
ble to physically implement logically irreversible transfor- 
mations of information in thermodynamically reversible 
ways. There are many practical reasons why a logically 
irreversible operation may not be thermodynamically op- 
timised, and it is clearly important and useful to explore 
such problems. In this paper, however, we are primarily 
concerned with the question: what is the fundamental 
limit for thermodynamically optimising the physical im- 
plementation of a given logical operation? 

We have demonstrated that, under the same conditions 
of uniform computing that imply logically deterministic, 
irreversible operations generate heat, logically indeter- 
ministic, reversible operations extract heat from the envi- 
ronment which can be converted into work. At the same 
time we have demonstrated that under other conditions, 
adiabatic equilibrium computing, information processing 
is able to progress without any exchange of work or heat, 
regardless of the type of logical operation. 

The thermodynamic reversibility of all logical opera- 
tions is, of course, based upon the definition of thermo- 
dynamic reversibility given in Sections IVI CI and IVI El 
Other approaches to thermodynamics (such as [J, [5j,[(j|]) 
use different concepts of entropy and correspondingly dif- 
ferent definitions of thermodynamic irreversibility to this 
paper. Ultimately the most important question is not 
what particular quantity one chooses to label as 'ther- 
modynamic' entropy. The GLP we have derived here is 
valid, whether one chooses to regard the Gibbs-von Neu- 
mann entropy as the true 'thermodynamic' entropy, or 
not. What is important is the actual work required to 
drive a system, the actual heat generated by that sys- 
tem. As there is no disagreement over the fundamental 
microscopic dynamics, it would be surprising if we were 
unable to be able to agree on these values, regardless of 
the definition of entropy to which we choose to adhere. 
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APPENDIX A: ONE BIT LOGICAL 
OPERATIONS 



P(/3 = 0|a = 0) = 1 

P(J3 = 0\a = 1) = 1 (A5) 

giving 

P(a = 0\P = 0) = p 

P(a = l|/3 = 0) = 1 -p (A6) 

This is logically deterministic and irreversible. As 
Va P(f3 — l\a) — the state f3 = 1 is not an output 
state of the operation and we leave it out of the table. 



1. Do Nothing: IDN 

The simplest operation is 



giving 



P((3 = 0\a = 0) 


= 1 




P((3 = 0\a = 1) 


= 




P(J3 = l\a = 0) 


= 




P(P = l\a=l) 


= 1 


(Al) 


P(a = O\0 = 0) 


= 1 




P(a = 0|/3 = 1) 


= 




P(a = 1 1/3 = 0) 


= 




P(a = l\/3 = 1) 


= 1 


(A2) 



and is logically deterministic and reversible. 



2. Logical NOT: NOT 

Logical NOT, acting upon an input bit with probabil- 
ity p of being in state 0, is very simple: 



4. Unset From Zero: UFZ(p) 

The reverse operation to RTZ, where the state is 
taken to state with probability p, will be called here 
the UNSET FROM ZERO operation. In Q this opera- 
tion was described in terms of the physical process that 
reverses LE, so was called 'Reverse Landauer Erasure' or 
RLE. In this paper we will refer to the logical operation 
as U FZ, and to the specific physical process that can be 
used to embody it as RLE. This operation may also be 
characterised as a random number generator. 



giving 



P(j3 = 0\a = 0) = p 
P(j3 = l\ a = 0) = 1 -p 



P(a = 0\/3 = 0) = 1 
P(a = 0|/3 = 1) = 1 



(A7) 



(A8) 



This is indeterministic but reversible. As V/3 P(a — 
l\/3) = the state a = 1 is not an input state of the 
operation and we leave it out of the table. 



giving 



P{(3 = 0\a = 0) 


= 




P{P = 0\a = 1) 


= 1 




P{P = \\a = 0) 


= 1 




P{P = l\a = 1) 


= 


(A3) 


P(a = O\0 = 0) 


= 




P(a = 0\P = 1) 


= 1 




P(a = 1 1/5 = 0) 


= 1 




P(a = 1 1/3 = 1) 


= 


(A4) 



This is logically deterministic and reversible. 



3. Reset To Zero: RTZ(p) 

If the input state occurs with probability p, then the 
RTZ(p) operation has the properties: 



5. Randomize: RND(p,p') 

The operation which takes an input probability of p of 
the state being and produces with an output proba- 
bility of p', regardless of input state: 



giving 



P(P 


= 0\a = 


0) 


= V 


P(f3 


= 0\a = 


1) 


= V 


P(f3 


= l\a = 


0) 


= 1 


P(f3 


= l\a = 


1) 


= 1 


P(a 


= 0|/3 = 


0) 


= V 


P(a 


= 0|/3 = 


1) 


= p 


P{a 


= l|/3 = 


o) 


= 1 


P{a 


= l|/3 = 


1) 


= 1 



p 

p' 



p 
p 



(A9) 



(A10) 
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This is indeterministic and irreversible. 

We note that RTZ(p) = RND(p, 1) and UFZ(p) 
RND(l,p). 



6. General One Bit: GOB(p, poo, pu) 

Finally, we consider the most generic operation possi- 
ble for 1 input bit and 1 output bit. The operation can 
be wholly defined by one input probability p and two 
conditional probabilities poo and p\\ 



so 



P(a = 0) 
P(a = 1) 
P(J3 = 0\a = 0) 
P(0 = 0\a = 1) 
P(0=l\a = O) 
P(/3 = l\a = 1) 



P 

1-p 
Poo 

1 - Pit 
1 - Poo 

Pll 



giving 



P(a 


= 0,0 = 


0) 


P(a 


= 0,/3 = 


1) 


P(a 


= 1,0 = 


0) 


P{a 


= 1,(3 = 


1) 



PPoo 

p(l -Poo) 
(l-p)(l-pn) 
(1 -p)pn 



and 



P{(3 
P{fi 



0) 
1) 



PPoo + (1 
P(l - Poo) 



P)(l-Pll) 

t-(l-p)pn 



P(a = 0\/3 = Q) 
P(a = 0\(3=l) 
P(a = l\(3 = 0) 
P(a = 1|/3=1) 



PPoo 


PPoo - 


h(l-p)(l-pn) 




p(l - Poo) 


p(l- 


Poo) + (1 - P)Pll 


(1 


-P)(l-Pll) 


PPoo - 


h(l-p)(l-Pu) 




(1 -p)pn 


p(l- 


Poo) + (1 - P)Pll 



(A14) 



(All) 



In general, this is logically indeterministic and irre- 
versible, but can become logically reversible or determin- 
istic under the right limits: 



(A12) 



(A13) 



IDN 
NOT 
RTZ(p) 
UFZ{p) 
RND{p,p') 



GOB(p,l,l) 
GOB(p, 0, 0) 
GOB(p, 1, 0) 
GOB{\,p,~) 
GOB{p,p',l-p') 



(A15) 
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